Diagnosing Problems

If your Intel hardware RAID system is beeping this almost always indicates a hardware fault. Faults need to be investigated and corrected promptly. The easiest way to do this is to always set up your servers or systems with the correct monitoring software when you install the system. This means that you do not need to find additional utilities or software when your system is in a potentially dangerous state.

Contact Support

Always contact Stone Support for warranty service and advice. We can help you when your system has a problem and help protect your data. The steps below are provided as a guide and are not exhaustive. They are based on a system with an Intel / Avago / Broadcom / LSI hardware RAID controller or module.

If your System is Beeping

• Note any warnings displayed onscreen
• Use the RAID monitoring software to check the health of your Logical or Virtual Drives (Virtual drives are the actual RAID arrays themselves, as opposed to each single physical drive).
  • Partially Degraded RAID6 arrays have had a single drive failure.
  • Degraded RAID5 arrays have also had a single drive failure.
  • Degraded RAID6 arrays have had two drive failures.
  • A RAID array marked as Failed has had more drive failures than it can tolerate.
    • In this situation, please contact Stone for additional support. We will take you through a process to try and recover the situation, by marking any Failed (or "Unconfigured Bad") drives as good and then scanning the controller for disk configurations that it can import from failed drives (known as foreign configurations). This is a temporary measure to bring the RAID array back online until replacement disks can be fitted. Usually, the last disk to fail can be brought back online and re-integrated into a failed RAID array as long as the drive is still workable.
    • If the last hard drive to fail is no longer detected, or cannot be brought back into the RAID configuration due to the number of defects or problems with the drive, then the RAID array has failed and the data will need to be restored from backups when the RAID array has been reconstructed.
• Check your servers Hot-Swap drive bays (if fitted) and look for any drive bays with an amber or flashing amber/orange LED. Amber usually means that the drive has failed; Amber/Orange usually means that the drive in that bay is being rebuilt. An Amber/Orange drive may not be the faulty drive in the system if that drive was previously unused and marked as a "Hot-Spare".
• If All Virtual drives are still healthy, and no single disks ("JBOD") are missing (for example, you can still access all disk volumes through File Explorer or Disk Management) then your system could have faults which include:
  • A controller fault, such as a memory fault or battery failure (check the RAID log for details)
  • Problems detected in a patrol read, such as bad blocked which indicated that a drive is about to fail - again check the RAID log for details but also check each physical disks Media Error Count and Predictive Failure Count.

• If the RAID system appears to be completely healthy, double check that the beeping noise is actually a RAID event. For example, some server systems may beep when a Power Fault is detected. Or, a UPS in the cabinet may have a problem. Check your servers warning triangle and ensure that it is green.

• Try silencing the RAID alarm. If the beeping persists, you have not got a RAID system fault.

Silencing the Alarm

If the RAID system beep is distracting your users, the alarm may be temporarily silenced. To do this:

• Open up the Physical or Logical Tab of Intel RWC2 / LSI MSM
• Right hand click on the controller.
• Then, left click on Silence Alarm

Replacing Hard Drives

Your Stone warranty service will carry out this for you, however some pointers are given below:

Do:

• Identify hard drives first before replacing them. Use the Drive location feature if you have hot-swap drive bays fitted.
  • Right hand click on the drive you want to identify, then left click on Start Locating Drive

• If your system doesn't include hot-swap drive bays, then record the Port or Slot number of the failed drive, as well as the serial numbers of either the failed drive, or, if this is not accessible, record the serial numbers of the remaining drives for elimination purposes. You can then work out which is the failed drive.
• Always hot-swap drives where possible. Do not shut the system down unless necessary - hot-swapping allows the rebuild process to start automatically if configured.
• Always replace drives with equal or greater capacity and equal or greater performance.
• It is recommended that you don't use Advance Format drives if the RAID array was built on non-Advance format drives.
• Always ensure that the rebuild process starts and then completes.
• If the rebuild process doesn't start automatically (this is the default on older controllers such as the Intel SRCSASRB and Intel SRCSATAWB) mark the replacement drive as a global hot-spare. This will trigger the rebuild process.

Applies to:

• Servers and workstations with a hardware Intel / Broadcom RAID controller or module

Background

The Intel RS2BL040 and RS2BL080 are 4 and 8 port controllers based on the LSI 2108 first generation 6G SAS chipset, and are Intel's branded version of the LSI 9260-4i and 9260-8i respectively. These cards have shipped in large numbers of Stone servers since 2012 to 2016. Therefore, it is useful to know how to replace failed cards and upgrade the controller firmware. The RS2BL080 is also a suitable replacement for the older SRCSASRB and SRCSATAWB controllers.

This article gives some more hints and tips specific to the RSB2L0x0 cards in addition to the Overview of the RAID Card Replacement Process.

Click on one of the links below for further instructions.

• I'm Upgrading to the RS2BL040 or RS2BL040 from an older RAID card
• I'm Replacing the same card like for like

I'm Upgrading to the RS2BL040 or RS2BL040 from an older RAID card

If you are upgrading from an older card, you need to follow the process below:

• If the operating system still boots on the older card, upgrade the RAID driver to the latest version to add support for the RS2BL040 or RS2BL080. For example, the Windows in-box LSI RAID driver supports older SRCSASRB cards but not the RS2BL080.
• Where possible or practical, a full system backup should be taken first.
• Once you've upgraded the driver, follow the process below for I'm Replacing the same card like for like.

I'm Replacing the same card like for like

Follow the process below:

• Remove old card; note the order of the drive cables and other signal cable connections.

• Check the old card for any Feature Keys which need to be moved to the new card.

• Move any RAID card battery to the new card if the old card had a battery unit fitted and the new card has not been supplied with a replacement, so long as the cards are the same model. If they are different models, a new battery will need to be requested.

• Disable Quiet boot in the BIOS.

• Fit the new card but leave the hard drive cables disconnected.

• Flash the latest approved firmware on the new card using the EFI Environment:

  • As of January 2016, the latest official Intel firmware for this card is 12.14.0-0185

  • If your replacement card has a version less than 12.12.0-0104, you may need to flash to version 12.12.0-104 first, especially if the flash program reports a bad flash image.

  • Engineers visiting a problematic server should also consider taking another "interim" version of firmware, 12.13.0-0154.

  • Finally, there is an updated RAID firmware available over and above what Intel have provided. Version 12.15.0-0239 can be downloaded from LSI directly, or an enhanced package which includes the flash tools for DOS, Windows and EFI can be downloaded here. This firmware may solve some instances where excessive "Illegal sense" messages are seen the logs, however this firmware has not been validated by Intel so it should not be used unless a resolution to a specific problem covered by the firmware release notes (included in the package) is required.

• Shut the system down; connect the drive cables.

• Boot the system up and follow the foreign configuration prompt to preview and then import the logical drives.

• Test the system. If a RAID driver installation or upgrade is needed and the system doesn't boot from the RAID (failing with a STOP 0x0000007b blue screen message) then a repair operating system installation may be needed.

Firmware Flash Methods and Examples

Applies to:

• Intel RAID Card RS2BL040 (ISRRAI-139)
• Intel RAID Card RS2BL080 (ISRRAI-129) / RS2BL080D (ISRRAI-134)

​

Instructions for the Windows Based Selviewer Tool

The Windows based Selviewer does not normally require rebooting to get the hardware log files, making it a useful tool. The instructions below are a guide as there are different versions of the Windows Selview tool and some of them look slightly different, or the filenames may vary.

Main Components

• IPMI driver which needs to be installed (some systems may have a Microsoft IPMI driver already installed in Device Manager, in which case you can use Device Manager to update the IPMI driver by pointing it to the Intel files)
• Some versions of Selviewer need Java to be installed but only if you want to run the full interface. If you just want to save events using the command line, you don't need Java. (look at the download page for your platform to see if you might need Java)
• The Selviewer tool itself

Quick Instructions - No User interface (recommended for System Admins or experienced users)

1. Download the Windows Based SELVIEWER Tool for your platform and extract it.

2. Go into the extracted Selview folder, and then into the x64 or x86 subfolder, depending on your operating system. Copy the contents of this x86 or x64 folder to C:\SELVIEW or another easy to type location.

You should end up with the following:

3. Open an Administrative command prompt and use the CD command to go to C:\SELVIEW\IMBDRIVER

4. Install the IPMI driver, usually using the command INSTALL C:\SELVIEW\IMBDRIVER

5. Some version of Selviewer require Java if you plan to use the full interface. As these instructions cover quick usage without the user interface, Java is not required to be installed.

6.  Using the Administrative command prompt, go to the folder containing Selview.EXE (use the corresponding sub-folder x86 or x64 depending on if your operating system is 32-bit or 64-bit).

7. Run SELVIEW.EXE sn.sel /save to save a copy of the events.

8. Send the output file to Stone support for review or alternatively use Wordpad or Notepad to examine them.

Short Instructions - Full Selview with User Interface

1. Download the Windows Based SELVIEWER Tool for your platform and extract it.

2. Go into the extracted Selview folder, and then into the x64 or x86 subfolder, depending on your operating system. Copy the contents of this x86 or x64 folder to C:\SELVIEW or another easy to type location.

You should end up with the following:

3. Open an Administrative command prompt and use the CD command to go to C:\SELVIEW\IMBDRIVER

4. Install the IPMI driver, usually using the command INSTALL C:\SELVIEW\IMBDRIVER

5. Install Java if your version of Selviewer requires it (you can also always tell this by seeing some .JAR files in the UI directory).

6.  Run the SELVIEW.EXE file, or the batch file (using the version which matches the edition of Java you installed (32bit or 64bit), if your Selviewer needs Java. For example run the x86\SELVIEW.EXE program if you have 32-bit Java installed or the x64\SELVIEW.EXE program if you have 64-bt Java installed.). You will need to run these as an Administrator.

7. Use the File > Save As option within the program to save a copy of the events, which you can then send to Stone for Analysis. You can also use Wordpad or Notepad to open them.

Detailed Instructions - Full Selview with User Interface

1. Download the Windows Based SELVIEWER Tool for your platform. This package normally comes with several utilities bundled inside, including the Selviewer tool.

2. Extract the downloaded file into a subfolder. Sample below.

3. If you are running a 64-bit system (most likely) go into the x64 subdirectory. If you are running a 32-bit system, go into the x86 subdirectory. This is important as the edition of the IPMI driver must match your operating system (32-bit or 64-bit).

4. Then copy the IMBDriver subdirectory by right hand clicking on it, then left hand click on Copy

5. Paste the copied folder to the root of the C drive, i.e. C:\

6. Open an Administrative Command prompt

The quickest way to do this on Server 2012 is to move the mouse cursor to the far bottom left of the screen, and then the Start image comes up, right hand click on it. Then choose Command Prompt (Admin).

7. Inside the command prompt, change to the folder containing the IPMI driver, by using the CD command. For example:

CD \

CD \IMBDRIVER

8. Then run the device driver installer:

INSTALL C:\IMBDRIVER

9.This should then report that the installation was successful. Note that in most instances a reboot is not actually required.

10. Go back to the files you extracted earlier.

11. If your version of Selview requires Java, this will need to be installed first. You can double check if Java is required by checking for the presence of .JAR files in the UI subfolder.

12. If required, Install Java. If not required, skip to step 13.

You will get either 32-bit or 64-bit Java depending on whether or not you are using a 32-bit or 64-bit browser to browse the internet. (64-bit browsers will require a 64-bit operating system).

It is important to note or realise which version of Java is being installed as you will need to use the matching Selviewer program. (While the IPMI driver needs to match your operating system, the Selviewer program needs to match the Java edition you have installed).

13. Now run the Selviewer application.

If your Selviewer doesnt require Java, then choose the version from the x86 or x64 folder which matches your operating system.

Right hand click on the Application and choose Run as Administrator.

14. If the Selviewer interface doesnt run, try running the batch file by using the Administrative command prompt; change to the location containing the files and run Selview. You should get an error message indicating the problem. For example, in the screenshot below, Java is not installed:

15. When the Selviewer interface has loaded, use the File and then Save As option to save the events to a file.

16. Send the event log file to Stone for analysis. You can also use Wordpad or Notepad to examine the file.

If the Selviewer program crashes when you run it, follow the instructions in the troubleshooting secion.

Troubleshooting - The Selviewer Program Crashes or Stops Responding

If the Selviewer program crashes or stops responding when you run it, the problem is likely to be around the IPMI driver. Check Device Manager for the presence of a Microsoft IPMI driver or an older Intel IPMI driver. If it exists, you will need to use Device Manager to manually upgrade the driver to the Intel version contained in the SELVIEWER package.

Re-running the "install" program for the IPMI driver will not solve the problem as this may not over-write an existing Microsoft IPMI driver.

Applies to:

• Server products that support SEL storage and retrieval

Problem

The wired network card in you some servers or workstations which include the Intel i350 network adapter may connect to the network slowly:

Symptoms include:

• The link speed may be reported as 10mbit, instead of Gigabit.
• The actual data throughput may be even slower than 10mbit, with speeds of only a few KB a second actually being transferred.
• This problem has been send on the Intel Server board S2600IP4 which includes four Intel I350 connections.

Cause

This appears to be a driver issue and has been found in driver version 18.3 through to version 20.1 (June 2015). Disabling advanced ethernet options such as support for Energy Efficient Ethernet (EEE) has no improvement.

This issue has only been observed in Windows 7 but likely applies to Windows Server 2008R2.

Work Around

Disable TCP/IP v4 Large Send Offload V2.

To do this:

• Open Device Manager.
• For each I350 LAN adapter:
  • Double click on it.
  • Go to the Advanced Tab.
  • Find Large Send Offload V2 (IPV4) in the list.
  • Set it to Disabled.
  • Click OK.
• Ensure you do this to all adapters in the system, especially if you are using any network teams.
• The adapter may briefly stop transmitting or receiving whilst you make this change.

Applies to:

• Stone Server and Workstation Products that include the Intel i350 Gigabit LAN Adapter running Windows 7 or Windows Server 2008R2.

Please find below a link to the Intel BIOS simulator for server systems.

This utility simulates the BIOS interface a user would normally see when pressing <F2> during POST. It can be used for support or research purposes.

Downloads

• Intel
• Alternative (latest as of 8/9/2014)

Applies to:

• Stone S1400 / S1600 / S2400 / S2600 server system families

Problem

• Add-in cards may not function reliably; for example network connections on add-in network cards may drop randomly or perform poorly.
• The system may be unstable.
• The motherboard hardware SEL Log records errors such as:
  • Critical Interrupt, PCIe Fat Sensor (#0x4)
  • CRITICAL event: PCIe Fat Sensor reports a fatal PCI Express Completion Timeout error

Cause

This can be caused on S1400 / S1600 / S2400 / S2600 Intel server or workstation boards by the use of an un-validated or unsupported PCI Express Gen.3 (generation 3) add-in card that has been fitted to a system that has a PCI Express Gen. 3 capable processor.

More Information

Intel originally released most of the above server platforms for PCI Express Generation 2 operation, namely for the first 32nm Xeon 2400 or 2600 series chips.

The new 22nm V2 versions of these chips support PCI Express Generation 3, which increases the data transfer rate. However, the system boards only support these increase data transfer rate on certain Intel validated add-in cards. The list of validated cards varies according to the Intel motherboard in use.

Resolution

Add-in cards which are not validated to run at PCI Express Gen. 3 on the affected server boards should have the appropriate PCI Express Slot configured for PCI Express Gen.2 operation only, by using the Processor PCIe Link Speed settings in the PCI Configuration menu.

See the following for more information from Intel:

• Technical Advisory
• Supported Adapters, Board and BIOS combination List at PCI Express Gen. 3

Intel Summary of Supported Adapters as of March 2015

Applies to:

• Stone S1400 / S1600 / S2400 / S2600 server system families.

How To Get A List of All Drives and Drive Health in Storage Spaces or Storage Spaces Direct (S2D) Pool

Existing Tools

Windows Server already gives you a number of ways of looking at physical disk health:

• Server Manager
• Windows Admin Center
• Powershell

However none of these tools give an easy way of looking at all of the disk health across nodes in an S2D cluster and storage pools, in an easy to use, single pane of glass. To do this, use the attached S2D-DiskInformaton

• Supports Storage Spaces
• Supports Storage Spaces Direct (S2D) and will show which disks are physically present in each node

Instructions

• Right hand click on the PS1 file and click "Run with Powershell"

Output

• Sample output below.

Applies to:

• Stone Server and Workstation Products

System Not Booting from a Hardware RAID Card

If your server is not booting from the RAID system - for example, the operating system does not appear and/or you get messages about the system trying to boot from the network, or possibly it the system boots into the EFI Shell - there are seven main possible reasons:

1. The main BIOS boot order may need adjusting. If the EFI shell is before the RAID Card in the Boot order the system will never boot from the RAID system. Check the BIOS boot order to ensure that the card is present and either at the top of the boot order list or near the top.
2. RAID Virtual Drive Failure - Is the RAID system beeping, indicating a drive failure? Are any of the hot-swap drive bays marked with an amber LED, indicating a possible drive fault? Use the RAID Web BIOS to check the RAID health. All virtual drives should be marked as "Online". If a Virtual Drive has Failed you may need to try and put a failed disk back online, or rebuild RAID array.
   • In this situation, please contact Stone for additional support. We will take you through a process to try and recover the situation, by marking any Failed (or "Unconfigured Bad") drives as good and then scanning the controller for disk configurations that it can import from failed drives (known as foreign configurations). This is a temporary measure to bring the RAID array back online until replacement disks can be fitted. Usually, the last disk to fail can be brought back online and re-integrated into a failed RAID array as long as the drive is still workable.
   • If the last hard drive to fail is no longer detected, or cannot be brought back into the RAID configuration due to the number of defects or problems with the drive, then the RAID array has failed and the data will need to be restored from backups when the RAID array has been reconstructed.
3. RAID Card Failure - do you see the RAID Card during the POST process? If you only see a system logo during the boot process you should check that the system's Quiet Boot feature is turned off to ensure that you can see POST messages.
4. Is the correct Virtual drive set as the boot drive inside the RAID BIOS? See Configuring the RAID Boot Volume.
5. A hard drive may be on the verge of failure. The RAID card may be in the process of failing the physical drive. Check each physical drives media error and predictive failure statistics in the RAID BIOS (both numbers should be 0). If you suspect RAID volume corruption as the result of a failing drive, replace the drive and rebuild the array. A RAID consistency check may also help, but use this option with caution - on a RAID1 or RAID5 system with a problem disk, single RAID parity information may not be enough to guarantee repairing the data. If you have a RAID6 based system, usually a disk consistency check will be more reliable. All disk consistency checks will take a considerable amount of time based on the volume size, number of disks, and RAID level.
   • Again, in this situation please contact Stone for further support.
6. If the RAID volume is healthy, and the RAID card is in the correct place in the main BIOS boot order, and lastly the right virtual drive is set as the boot drive (or if you only have one volume) then disk corruption may be a possibility - if you recently experienced a system crash your Windows volumes may be corrupt. You may be able to boot from operating system installation media and run a CHKDSK /F on your Windows boot volume. If that does not help, you may need to perform a repair installation or restore from backups.
7. Is the server warning symbol lit? It should be solid green during normal operating. If it is orange, contact Stone Support for assistance, and/or retrieve the hardware system event log (SEL).
8. Are you trying to boot from a virtual drive that is 2TB in size, or larger? GPT partitioning has to be used to boot from a volume that is 2TB in size, or large. To boot from a GPT disk, you will need UEFI support or UEFI optimised boot enabled.

In all situations please note any error messages or other symptoms and contact Stone support for further assistance.

Some Resolutions to the Above

Disabling Quiet Boot

This will enable you to see all of the POST messages and see the RAID BIOS start.

• Tap the F2 key during the server post process to go into the main BIOS setup. Usually you should start tapping the F2 from the time you hear the first POST beep until you see a message confirming that the system will enter setup.
• Ensure that in the Main menu, Quiet Boot is set to Disabled.

Checking the BIOS Boot Order

Use this process to check that the main BIOS is attempting to boot from the RAID card, or to check that the RAID card is being identified by the system. If the RAID card does not appear as a boot option, the card may have failed, its BIOS may not be enabled, or more likely its virtual drive has failed meaning that the RAID card has no bootable device to present to the main BIOS.

The instructions below are based on an Intel S5520HC system but are applicable to most modern Intel based server systems.

• Tap the F2 key during the server post process to go into the main BIOS setup. Usually you should start tapping the F2 from the time you hear the first POST beep until you see a message confirming that the system will enter setup.
• Go to the Boot Options menu
• Check the Boot Options, or Boot Devices, for the RAID card.
• In the example below, the RAID card is not in the list of Boot Options and the system will not boot from it.

If the RAID card is not in the list of options:

• Go into Hard Disk Order

• From the Hard Disk order menu, see if the RAID Card is present. Look for an LSI or Intel device.

• Either use the arrow keys to highlight the Intel / LSI device and then use the + or - keys to move the Intel device to the top, or press Enter on Boot Option #1 and then in the submenu that appears select the RAID controller as the first boot device by highlighting it with the arrow keys, and then press Enter again.

• Your Hard disk Boot Options order should then look something like this:

• Press ESC to return to the main Boot Options screen and confirm that the RAID controller is now shown in the list.

​

• Press F10 to accept the changes - you will be prompted to Save the changes and reboot the system.

Configuring the RAID Boot Volume

• Enter the RAID BIOS when prompted (usually CTRL+G on Intel controllers or CTRL+H on LSI controllers)

• Press Start (after selecting the right adapter if you have more than one adapter in the system).

• Click on Virtual Drives

• You can then see the current boot virtual drive. In most instances this is normally boot drive 0, or VD0, as this usually contains the operating system (although this can vary!).
• If you need to change the boot drive, click on "Set Boot Drive" and then click on Go.

• Once you have selected the radio button, click on Go to make the change.

Applies to:

• Systems with an Intel or LSI Hardware RAID Controller or Module.
• Systems based on an Intel server platform

Problem

• Some Xeon E5 workstation or server based systems may not have all PCI Express Slots available for use.
• When you plug a card into a Slot that is not available, the card is not detected at all in the BIOS or in Windows.

Cause

In a modern workstation or server system, the PCI Express Slots are almost without exception connected to the PCI Express Lanes on the system processors. This enables fast transfers and low latencies. Most Xeon E5-2400 or Xeon E5-2600 systems support up to two CPU sockets. Many (but not all) of these systems split the PCI Express slots in the system between the two processors.

This means that if you have a dual CPU socket system, but only one processor is fitted, then some PCI Express Slots may be unavailable.

Resolution

• Refer to the motherboards technical product specification (TPS) to see which slots are connection to which processor. Generally speaking, the PCI Express slots nearest to the system memory are available, as they run from Processor 1 on most motherboards. The PCI Express slot furthest away from the system memory may also be available, however this will be a PCH Slot and may suffer from increased latency.
• Use the Intel Support Page to find TPS documents for Intel Server boards or see the attached article for some commonly used board TPS documents.
• Fit add-in cards only to slots which are available.

More Information

Sample Layout Diagram 1 - Intel S2600CW

The first diagram shows the Slot availability as part of the chipset and processor layout; the second diagram shows the availability by physical slot.

Sample Layout Diagram 2 - Intel S2600WT in an Intel R2308WTTYS chassis

The first diagram shows the Slot availability as part of the chipset and processor layout; the second diagram shows the availability by physical riser slot.

The three riser diagrams from the System's Technical Product Specification show the PCI Express availability at each slot, and which processor provides the connectivity.

Terminology

PCH Slots

PCI Express Slots connected to the PCH are usually x4 or x8 PCI Express slots connected to the motherboard chipset. Because they don't have a direct connection to the processor, the performance of these slots is often slightly lower. However they will always be available for use in whichever processor configuration.

MUX Slots

MUX Slots are PCI Express Slots that share some or all of the PCI Express lanes with other slots or devices. For example, a PCI Express x8 MUX based slot might have four lanes coming directly from the processor, and four more lanes that go via a MUX. The MUX shares the PCI Express Lanes between that slot and another slot.

MUX based slots usually do not have any compatibility issues but bear in mind that the bandwidth on MUXPCI Express lanes is shared.

DMI Slots

PCI-DMI Slots are PCI Express Lanes running from a CPU. However, these lanes are normally used for CPU to Chipset communications. When implemented as PCI Express Lanes, the result is often a PCI Express slot that is slower or smaller, for example Gen.2 and x4.

The Difference between Physical or Mechanical, And Electrical

Some PCI Express Slots may not be fully wired. For example, a PCI Express x16 connector may, on some systems, only be x8 electrically wired. This means that half of the PCI Express lanes are unavailable. Plugging in a PCI Express x16 device may work if the device supports running with only half of the lanes connected, but it will run at reduced bandwidth.

PCI Express Generations

There are different generations of PCI Express available. PCI Express Generation 3 (or Gen.3) supports 8 Giga-tranfers per second, resulting in a little less than 1GByte a second of transfer performance per lane.

If you have a PCI Express Generation 3 device, always use PCI Express Generation 3 slots in preference to PCI Generation 2 slots wherever possible.

Applies to:

• Stone Xeon E5 based server system families, including S2400 and S2600 servers.

Please find below a link to the Intel BIOS simulator for S1200KP server systems.

This utility simulates the BIOS interface a user would normally see when pressing <F2> during POST. It can be used for support or research purposes.

Downloads

• Intel
• Alternative (latest as of 8/9/2014)

​

Applies to:

• The Stone S1200KP server family

Problem

On some combinations of hardware RAID card and server or workstation motherboard, when you attempt to go into the RAID Card BIOS Console the system ignores the request and boots normally.

For example, you press CTRL+G during the RAID initialisation screen when an RMS25CB080 is fitted, but unexpectedly the RAID Card BIOS Console does not start when POST completes.

Cause

This is normally caused by a compatibility issues between the RAID card or module and the motherboard.

Resolution

Both a resolution and workaround are available for this issue. For the workaround, scroll down.

To resolve the problem, ensure that both the motherboard BIOS and RAID BIOS are up to date. Usually this can be done on most modern motherboards using the EFI environment, meaning that you don't need to access the RAID BIOS Console to actually perform the update.

Workarounds

Two work arounds are available - Work Around 1 uses the Boot Menu (F6) and Work Around 2 uses the BIOS Setup (F2).

Work Around 1 - Using the Boot Menu (F6)

Do the following:

• Use CTRL+G as normally (or the correct combination for your RAID card) to request the RAID Card BIOS Console as normal.
• In addition, enter the motherboard boot menu. This can usually be done before and/or after the RAID Card initialisation screen has been displayed. On most Intel server boards, the boot menu key is F6.
• Once the motherboard boot menu appears, choose PCI ROM Setup.

​

• ​The RAID BIOS Console is then displayed.

​

Work Around 2 - Using the BIOS Setup (F2)

Do the following:

• Use CTRL+G as normally (or the correct combination for your RAID card) to request the RAID Card BIOS Console as normal.
• In addition, enter the motherboard BIOS setup. This can usually be done before and/or after the RAID Card initialisation screen has been displayed. On most Intel server boards, the BIOS setup key is F2.
• Once the motherboard BIOS appears, go to the Boot Manager Screen.
• Choose PCI ROM Setup.

• ​The RAID BIOS Console is then displayed.

​

​Applies to:

• All Intel Server and Workstation systems with Hardware RAID cards or modules fitted.

​

Problem

When you install Windows 7 or Windows Server 2008R2 on affected platforms the system performance can be noticeably poor. The mouse cursor is slow to respond; the system appears to be busy or have high CPU usage, and screen updates can be slow.

Affected Systems

Systems affected include:

• Servers based on the S5500 and S5520 platform, including the S5520UR, S5520HC, S5520HCR, S5520HCT, S5520SC and S5500BC
• Server systems based on the S2600UR platform (which include the S5520UR motherboard)
• Desktop boards based on the X58 PCH, including the Intel DX58SO, DX58SO2 and DX58OG

Cause

The root cause is the Microsoft provided Intel LAN controller driver which ships with Windows 7 and Windows Server 2008R2. This driver does not properly support the controller and it will cause the above symptoms.

Resolution

Update the driver with the latest Intel LAN driver available from the Stone driver finder or Intel website. Updating the driver will restore the lost performance and also add the additional features provided by the full Intel LAN driver package.

The LAN driver for these systems should be date 2011 or newer, as opposed to the Microsoft 2009 in box driver.

Applies to:

• Desktops and Servers that use the Intel X58 PCH

Please find below a link to the Intel ESRT-2 RAID BIOS simulator for server systems.

This controller is integrated into a number of onboard SATA or SAS controllers to provide basic RAID functionality. Most implementations of this controller provide basic hardware or firmware assisted RAID.

ESRT-2 RAID should not be confused with RST-e RAID. Intel Rapid Storage Technology - Enterprise - is a software based RAID solution with some additional logic and firmware build into the motherboard PCH chipset and BIOS in most cases.

This utility simulates the RAID BIOS interface a user would normally see when pressing <CTRL><G> or <CTRL><E> (as dependant on model and BIOS message) during POST. It can be used for support or research purposes.

Downloads

• Intel
• Alternative (last updated 8/9/2014)

Applies to:

• Stone S1200 / S1400 / S1600 / S2400 / S2600 server system families

Limited SEL Log Storage Space

The hardware event logs retained by the motherboard's Baseboard Management Controller (BMC) are stored in a limited capacity flash memory. When this memory has been filled, you will usually find that SEL events are not stored. For example, if you save a SEL log and find that is has a few thousand entries, but the newest entry was some time ago, then it is likely that the SEL log is full.

How To Clear the SEL Log

Clearing the SEL log in the motherboard's memory will allow new events to be stored.

To clear the SEL log, use the SELVIEW utility, either from the Windows Command line or Internal EFI Shell - (or, for the older S5000 based server boards, use the SELVIEW utility from DOS instead of EFI).

Simply type in SELVIEW /clear and then follow the confirmation instructions to clear the log.

Applies to:

• Desktop and Server systems that support SEL storage and retrieval.

Problem

On some Intel servers a glitch with the RAID Copyback feature can cause the RAID Card to continue to beep even after a failed drive has been replaced. The system may start beeping again when the system is rebooted.

Background

The Intel RS2BL040 and RS2BL080 are 4 and 8 port controllers based on the LSI 2108 first generation 6G SAS chipset, and are Intel's branded version of the LSI 9260-4i and 9260-8i respectively. These cards have shipped in large numbers of Stone servers since 2012 to 2016. There is a wide variety of firmware versions in use in the field. During the card's life, new features have been added by Intel including the Copyback feature.

Problems with the Copyback Feature

The Copyback feature is designed to rebuild back onto the original drive slot after a RAID repair to a hot-spare has been completed. For example:

Pre-Failure state:

• Slot 0 - Virtual Drive 0 - RAID 1
• Slot 1 - Virtual Drive 0 - RAID 1
• Slot 2 - Hot Spare

State after drive failure:

• Slot 0 - Virtual Drive 0 - RAID 1
• Slot 1 - failed
• Slot 2 - Rebuild drive - Virtual Drive 0 - RAID 1

​With Copyback, the idea is that when you replace the failed drive in Slot 1, that the system rebuilds onto Slot 1, and Slot 2 reverts to being the hot-spare drive.

In some versions of Copyback, notably with firmware version 0104, when you replace a failed drive in a system that does not have a hot-spare, the act of marking a new drive as a hot-spare to begin the rebuild process leaves the system requesting a second rebuild after, to complete Copyback.

Recommendation

To avoid a copyback issue on the RS2BL040 or RS2BL080 with firmware version 0104, either change the RAID card properties so that it automatically rebuilds, disable Copyback, or upgrade the Controller firmware before replacing the drive.

To Set Automatic Rebuilds On

You will need the Command line utility Storcli. This can be used inside EFI or a Windows environment. Use the commands below to list the controllers and get the right controller number, i.e. 0 or 1. For example, you may have an onboard ESRT2 controller - usually controller 0 - and an add-in RAID module or controller, usually controller 1.

storcli show

To show the existing auto-rebuilld setting:

storcli /c0 show autorebuild

To enable auto rebuild on controller 0:

storcli /c0 set autorebuild=on

To Disable Copyback

Again, you will need the Command line utility Storcli. This can be used inside EFI or a Windows environment. To disable Copyback on controller 0:

storcli /c0 set copyback=off type=all

To Check That Copyback is Disabled

Run the command below:

storcli /c0 show copyback

To Upgrade the Firmware

See the attached article.

What If I Have Already Replaced the Drive and my System Beeps?

First, download the RAID TTY log using RAID Web Console to confirm the cause of your issue. If Copyback is the cause you of your problem, you will need to either:

1. Fit an additional drive and re-create the hotspare. Then delete the hot-spare.
2. Or alternatively, if you don't have a hot-spare configured/available:
   • Use the StorCLI command as above to disable copyback (storcli /c0 set copyback=off type=all)
   • Remove the drive that you recently replaced.
   • Wait 60 seconds.
   • Refit the drive again.
   • Clear the Foreign configuration from the drive you just re-inserted (Note: Do not try and make this drive "online").
   • If the system doesn't automatically rebuild, mark the drive as a Hotspare again and then check that the system is rebuilding.
3. Or, as a last resort, if the system has no room for an additional drive, you will need to back up the virtual drive, re-create the virtual drive and then restore the data.

More Information

See this Intel article.

Applies to:

• Intel RAID Card RS2BL040 (ISRRAI-139)
• Intel RAID Card RS2BL080 (ISRRAI-129) / RS2BL080D (ISRRAI-134)

​

Bootable Pen Drive

There are many instances where a "DOS" or Windows 98SE bootable pen drive is required - for example, some BIOS updates or other low level updates can only be done in this way. (Stone is working to provide Windows update utilties wherever possible, but these arent always available yet).

If you need to create a DOS or Windows 98SE (or FreeDOS) bootable equivalent, please see some links to third party sites below which host various solutions.

Rufus - http://rufus.akeo.ie/

HP USB Format Tool - Download here

Windows 98 Boot files - here

Applies to:

• All desktop, laptop and server products

Please find below a link to the Intel RAID Simulator for 12G (12-Gbit) adapters on server systems.

These controllers are available as either a plug in PCI Express Module, a PCI Express Card, or Integrated onto the motherboard. These controllers should not be confused with onboard software RAID or SCU controllers.

This utility simulates the RAID BIOS interface a user would normally see when pressing <CTRL><R> during POST. It can be used for support or research purposes.

Downloads

• Intel
• Alternative (last updated 2/2/2016)

Applies to:

• Intel 12G Hardware RAID Controllers or modules, including RMT3PB080 (ISRRAI-160), RS3DC080 (ISRRAI-167), RMS3CC040 (ISRRAI-171), RMS3CC080 (ISRRAI-172), RS3DC040 (ISRRAI-174) and RS3DC080 (ISRRAI-175).

The Hardware System Event Log (SEL) - Introduction

The hardware SEL is a function of most server motherboards. The baseboard management controller (BMC) records hardware events, including most health events, into an area of the motherboard non-volatile memory. In other words, this event log can be retrieved even after the system has been cold booted. A CMOS clear is likely to delete all of the events.

The hardware SEL should not be confused with the Windows System event log, which records Windows events. The hardware SEL will record things such as power events (power on, power off/loss of power, power redundancy events etc.) and faults such as fan issues, memory ECC events, or some motherboard errors such as PCI Express bus faults.

Choosing the right method of retrieving the SEL information

Motherboard manufacturers such as Intel provide utilities to display or save the SEL log. Utilities available depend on the motherboard model. Use the DOS or EFI mode version when Windows is not operating. For most models, a Windows version is available which allows SEL events to be retrieved while Windows is running, after the installation of an IPMI management driver.

While DOS mode tools are available for some platforms, use of EFI tools is preferred as you do not need to prepare a Windows 98SE DOS bootable pen drive.

Methods available for retrieving the SEL information

You will need to download the SEL viewer utility for either EFI, DOS or Windows. Always consult the motherboard or platform download pages for updated downloads.

On all Windows systems you need to install the correct package (32-bit / 64-bit) in order to install the correct IPMI driver. The IPMI drivers are included in the distributions. Note that the Window's SEL log utility requires Java to be installed if you want the GUI displayed. Using the command line option to save the log does not require Java.

Applies to:

• Workstation and Server systems that support SEL storage and retrieval.

Introduction

The SysInfo tool (System Information Retrieve Utility) is a utility available for most S2400 and S2600 systems. It records the hardware SEL Log but additionally lots of other useful diagnostic information.

When trying to debug an issue, it is usually easiest to retrieve the basic SEL log. If this does not reveal the cause of the problem - and you definitely have a hardware problem, for example the system warning does not show a solid green indication - then you may need to use the Sysinfo tool to gather additional information.

Download the SysInfo tool from the motherboard or Server systems download page.

For example:

S2600 Family - Tool is available here.

M50CYP Family - Sysinfo tool is available here.

SysInfo Tool Instructions

Windows

• As per the standard Windows SELViewer tool, download the tool package and extract it to a folder on the system.
• Again, as per the standard SELViewer tool instructions, using an Administrative command prompt, Install the IPMI driver.
• Go into the Sysinfo folder.
• Run sysinfo.exe –ni
• This will generate two or three files which need to be sent to Stone support
  • Sysinfo_log.txt, RAID_NVRAMlog.txt and PCI_log.txt in some versions of Sysinfo
  • A seperate Logfiles directory in other versions of Sysinfo
  • (Sort by date and time to find the folder thats been created, or files created)

UEFI

• As per the normal SELVIEWER instructions, download and extract the tool package, and then copy the contents of the  UEFI_sysinfo_Vxxx to a folder with a short, simple name on your USB pen drive.
• Plug the pen drive into the server
• Reboot into the BIOS (using the F2 key to go into the BIOS during POST)
• Go to the Boot Manager screen
• Boot into the Internal EFI Shell
• Type in fs0: and then Enter to go to the pen drive
• Type in CD <short folder name>
• Type in  sysinfo.efi –ni
• This will generate three files which need to be sent to Stone support - sysinfo_log.txt, RAID_NVRAMlog.txt and PCI_log.txt.

System Event Log Troubleshooting Guide

Intel have made available a guide to help interpret Selviewer and Sysinfo information. The guide is available here, or attached.

This guide allows you to decode the hexadecimal information of events recorded in the Sysinfo log to determine, for example, a problem sensor or reading.

Applies to:

• Desktop and Server systems that support SEL storage and retrieval.

Issue

Entry-level servers that use basic Serial-ATA hard drives may experience slow disk transfers if the server is a Microsoft Windows Domain Controller (DC). Typically file transfers to the server over the network are slow (around 10MB/second or less). File transfers between hard drives or partitions on the actual server itself are also slow.

Hard Drive Caching Features Disabled by Design

This issue is caused by the deliberate disabling of hard drive caching features on any system that is a domain controller. A Windows Domain Controller will attempt to disable hard drive caching features on every boot. Typically Windows cannot succeed in disabling caching features on systems equipped with a full hardware RAID controller, but it can succeed in disable caching features on systems equipped with single SATA hard drive.

Re-Enabling Hard Drive caching Features

This setting is visible in Device Manager of the affected system, however on every system boot the features will be disabled again. Both check boxes need to be ticked to restore maximum performance. (The default on any non-Domain Controller system is that at least the top check box is ticked).

To re-enable full hard drive caching features on every boot, use the attached package.

• Use fixdc-diskcache.zip - for IDE or non-SCSI disks only, targeted for use with Server 2008R2.
• Use fixdc-diskcache2012r2.zip - for all types of disk including SCSI, targeted for use with Server 2012R2.

Both fixes use the same utility but with different options. They are both usable on 2008R2 and 2012R2. However the 2012R2 version has been configured to enable caching features on all disks, including SCSI disks. This is needed on modern 2012R2 servers as most IDE or SATA disks now work through AHCI controllers which often, when the right driver has been loaded, appear as SCSI disks to the system. The 2008R2 fix only configures disks that are not recognised as SCSI to the system.

Instructions

• Extract the contents to C:\Regfix
• Use FIX.BAT to turn the caching features on

• Use a Scheduled task to call FIX.BAT on every system startup with a 30 second delay.

The package includes an importable Task Scheduler XML file; however you will need to change the task privledges and use your own Administrator username and password. If you want to use this XML file, you will need to extract the contents of the package to C:\Regfix

Applies to:

• Any Windows 2008R2 or Server 2012R2 Domain Controller Server using a basic Serial ATA (SATA) hard disk.

Problem

• RAID Card does not appear to be detected
• The system only displays the Intel splash screen and does not display the RAID Card BIOS (i.e. the RAID card list of drives)
• When you attempt to go into the BIOS using the F2 key, the system does not respond and appears to hang

Cause

This is caused by the Intel Quiet boot setting on the server board BIOS. This causes the Intel splash screen to be displayed; all RAID and system error messages are hidden behind it. This means that any RAID card prompts (i.e. something that requires a key-press or some action) causes the system to appear to hang, as you never see the instructions. The system will not enter the BIOS until the RAID card BIOS completes.

Scenario

This can happen either after motherboard or RAID card replacement, or on a system that has been running fine but has experienced a RAID event (such as a failed volume, card fault or RAID battery issue) causing the RAID card to start to need corrective action during the boot process.

Resolution

In this situation you may find that waiting for the BIOS setup to appear may happen after a considerable wait - for example, after the RAID card times out - but often this is not the case. You need to disable Quiet boot in the BIOS but in order to gain access to the BIOS you will need to remove the RAID card from the system.

Disable Quiet Boot

This will enable you to see all of the POST messages and see the RAID BIOS start.

• Remove the RAID card from the system. This should only be performed by an experienced technician. You may be able to remove the card but leave the drive and other signal cables connected. If you do so, place the card on an ESD safe non-conductive surface. Remember that cards with cache batteries fitted will be partially powered even when the system is turned off.
• Tap the F2 key during the server post process to go into the main BIOS setup. Usually you should start tapping the F2 from the time you hear the first POST beep until you see a message confirming that the system will enter setup.
• Ensure that in the Main menu, Quiet Boot is set to Disabled.

Applies to:

• Systems with an Intel or LSI Hardware RAID Controller or Module.
• Systems based on an Intel server platform

Setting the Boot Virtual Drive (VD)

Hardware RAID cards support multiple virtual drives. For example, you may have a RAID 1 volume for the operating system, and a RAID 6 volume for your data. These will likely be identified as virtual drives VD0 and VD1. In a similar way that the main motherboard BIOS can be set to boot from different devices - including the RAID controller - the RAID controller itself can be set to boot from different virtual drives.

If you have moved or recovered a virtual drive, you may need to configure the controller to boot from the correct drive. An indication that this is require is if the RAID card is seemingly ignored in the boot order, or if you get a BIOS message about an invalid boot device, or an invalid boot loader.

The instructions vary depending on whether or not you have a 3G or 6G controller, or one of the new 12G controllers.

• Instructions for 3G / 6G Controllers (Graphical-mode RAID BIOS)
• Instructions for 12G Controllers (Text-mode RAID BIOS)

Instructions for 3G / 6G Controllers (Graphical-mode RAID BIOS)

• Boot the system into the RAID BIOS by Pressing CTRL + G when prompted.

• After the system has detected the drives, the RAID BIOS will start. You will be prompted to confirm which controller you want to access.

• Press Enter or click on Start.
• Note the IDs and sizes of the virtual drives - for example, ID: 0 - 278GB, ID: 1 - 2.725TB. Usually an operating system is installed on the smaller RAID 1 volume.

• Click on Virtual Drives on the left hand side.

• You will now be presented with a list of virtual drives and can now set the boot drive.

• Highlight the VD you want to boot from. On most systems, this is the smaller drive used for the operating system.
• Then select Set Boot Drive, then click on Go.

• You will be returned to the same screen. Note that the system has accepted the change in boot VD. Click Home to exit making changes to Virtual Drives.

• In the main view, click on Exit.

• Select Yes to exit the RAID BIOS.

• Reboot the system using CTRL+ALT+Delete or the reset button when prompted.

Instructions for 12G Controllers (Text-mode RAID BIOS)

• Boot the system into the RAID BIOS by Pressing CTRL + A when prompted.

• After the system has detected the drives, the RAID BIOS will start.
• Note the IDs and sizes of the virtual drives - for example, ID: 0 - 278GB, ID: 1 - 2.725TB. Usually an operating system is installed on the smaller RAID 1 volume.

• Press CTRL+N twice to go to the Ctrl Mgmt tab.

• Use the TAB key to highlight Boot Device.

• Press Enter to get the list of VDs available for booting.
• Highlight the VD you want to boot from. On most systems, this is the smaller drive used for the operating system.

• Now use the TAB key again to go through all of the options on the page, until APPLY is highlighted. Press Enter on APPLY.

• Once the setting has been applied, the screen will simply highlight the first option on the page again. There is no other confirmation that the settting has been saved.

• You can now safely exit the RAID BIOS by pressing ESC and  then pressing Enter whilst OK is highlighted.

• Then reboot by using CTRL + ALT + Delete when prompted.

Applies to:

• Intel 12G Hardware RAID Controllers or modules, including RMT3PB080 (ISRRAI-160), RS3DC080 (ISRRAI-167), RMS3CC040 (ISRRAI-171), RMS3CC080 (ISRRAI-172), RS3DC040 (ISRRAI-174) and RS3DC080 (ISRRAI-175).
• Intel 6G Hardware RAID Controllers or modules, including Intel RAID Card RS2BL040 (ISRRAI-139), Intel RAID Card RS2BL080 (ISRRAI-129), RS2BL080D (ISRRAI-134), Intel RAID RMS25CB080 (ISRRAI-151)

The Hardware Event Log

Most modern servers retain hardware events, including informational, warning, and error events, in a dedicated hardware event logging system held on the motherboard. These logs can be useful in troubleshooting problems, such as a flashing hardware health symbol.

With Intel based servers from the 2010 model year and newer, it is possible to retrieve these logs using the following methods:

• The EFI shell (which can only be used when the host operating system is not running) and the EFI Selviewer tool.
• Using the Windows Selviewer tool when Windows is running directly on the system.
• Accessing a Remote Management Module, if fitted, using a web browser

It is also possible to retrieve a basic hardware event log directly using the VSphere client for servers running the VMWare ESXi Hypervisor. The method to achieve this is below.

Instructions

Steps:

• Log into your VMWare client and connect to your VCenter server, or directly to your host server.
• Click on the host you want to examine
• Go to the Hardware status tab

• Examine the hardware sensors and ensure that all sensors are showing as "Normal".

• Then, using the "View" dropdown, select "Alerts and Warnings".
• Ensure that the system reports that there are "No alerts or warnings".

• Finally, using the "View" dropdown, select "System event log".
• Scroll down on the right hand side to view the most recent events
• Use the Export option on the top right hand side to save a copy of the events. This file can then be sent to Stone Support for review.

Applies to:

•  Server systems that support SEL / IPMI storage and retrieval.

Please find below a link to the Intel RSTe RAID BIOS simulator for server systems.

Intel Rapid Storage Technology - Enterprise - is a software based RAID solution with some additional logic and firmware build into the motherboard PCH chipset and BIOS in most cases.

This utility simulates the RAID BIOS interface a user would normally see when pressing <CTRL><I> during POST. It can be used for support or research purposes.

Downloads

• Intel
• Alternative (last updated 23/2/2016)

Applies to:

• Most Stone Servers from the Families including S5000, S5600, S1200, S2400, S2600

Short Instructions

1. Download the EFI SEL Viewer and extract it to a FAT32 formatted USB pen drive. Connect the pen drive to the system while it is powered off.

2. Boot the server into the EFI Shell  by using the F2 key during POST to go into the BIOS and then by using the Boot Manager screen to access the Internal EFI Shell

3. Change drive to the pen drive by typing in fs0: and then Enter. Change to the folder that you extracted the SELVIEW program to by using the cd command.

4. Run the SELVIEW program to run the GUI or run selview /save sn.sel to save the event log entries to a file.

5. Send the log file to Stone support for analysis.

6. If you need to clear the SEL Log of all entries, use selview /clear

Detailed Instructions

These instructions involve using the EFI shell for diagnostics. This can only be performed when the operating system such as Windows or ESXi is not running. If you need a Windows based tool, this is also available.

1. Download the EFI SELVIEWER tool for your motherboard or server platform.

Many downloads contain tools for EFI and Windows, in which case find the folder which references UEFI:

2. Create a folder called SEL on a FAT32 formatted USB pen drive

3. Copy the SEL Viewer files to the SEL folder on the USB pen drive

4. Boot the system into the EFI Shell and follow the instructions to get to the fs0: prompt.

5.Use the cd command to change to the SEL directory.

(6. Optional - use the DIR command to list files. Executables are highlighted in bright green).

7. To run the SEL Viewer and just save the log entries to a file on the pen drive, without opening the GUI, type in:

selview /save sn.sel

And press Enter. Replace sn.sel with the serial number of the system.

8. To run the SEL Viewer GUI, just type in selview and press enter.

9. To save the event log to a file using the GUI, do the following:

• Press Enter to access the file menu and then use the down arrow key to highlight "Save As"

• Press Enter. You will then be asked for a filename. Give a filename and press Enter again.

10. When you have saved a copy of the SEL Log to the pen drive, reboot the system.

11. Send the SEL text file into Stone support for analysis.

12. If you need to clear the SEL Log of all entries, use selview /clear

Applies to:

• Desktop and Server systems that support SEL storage and retrieval.

Introduction

The new 12-Gbit SAS Hardware RAID Controllers use the new LSI / AVAGO 12Gbit SAS Controller Chipset, the 2308. Cards based on this chipset feature an updated RAID BIOS that reverts to a more text-based layout. The main functionality of the previous RAID Web BIOS is still available, but accessing it is different.

When Do You Need to Import a Foreign Configuration?

A "foreign configuration" situation occurs when the configuration on the RAID controller doesn't match the configuration on the drives. This often happens if the RAID controller has just been replaced, or if the physical drives have been moved from one server to another.

Many controllers with up to date firmware may automatically import foreign drives if there wasn't a configuration previously on the card. If this automatic import doesn't happen, or if your controller has been previously used, you will need to follow the instructions below.

Instructions:

• If the controller is new:
  • Don't connect any drives until you have upgraded the RAID firmware.
  • Download the latest controller firmware from the Intel web site.
  • Extract the firmware to a folder inside a FAT32 formatted USB pen drive.
  • Run the Firmware update NSH file to start the upgrade process.
• With the system off, connect your drives to the controller. Ensure all drives are connected, and that you plug the drive cables into the correct ports. Example below.

​

• Boot the system into the RAID BIOS by Pressing CTRL + A when prompted. If you get a message with regards to a foreign configuration being detected, you may press F to try and import the foreign configuration however it is usually best to go into the RAID BIOS and check that this has been done successfully.

• If the controller has no valid configuration, you will be greeted with a RAID screen that shows no configuration is present, although there is a reminder on the right hand side that there is Foreign Config information available.

• To get to the Foreign View menu, press CTRL + N four times to move through the tabs at the top.

• Notice in the example above that there is one Virtual Drive (VD) in the foreign configuration. Check that all of your configurations or virtual drives are present. If not, use the PD Mgmt tab to check that all of your physical drives are present as expected. If drives are missing, switch the system off, and re-check all cables and drive seating.
• To import the foreign configuration, use the up arrow to select the RAID controller.

​

• Now press F2 to perform Operations on the Controller. You will get a pop-up menu for Foreign Config. Press Enter to open the Foreign Config menu to be given the choices of Import or Clear.

• Make sure Import is highlighted, and press Enter.
• You will then be prompted if you are sure you want to do this.

• Highlight Yes and press Enter.
• The system should now go back to the VD Mgmt menu. Note that your expected VDs are online.

• You can now safely exit the RAID BIOS by pressing ESC, and then reboot by using CTRL + ALT + DEL.

• During normal boot, after drive detection, the system should confirm that 1 Virtual Drive (VD) is installed and operational.

Applies to:

• Intel 12G Hardware RAID Controllers or modules, including RMT3PB080 (ISRRAI-160), RS3DC080 (ISRRAI-167), RMS3CC040 (ISRRAI-171), RMS3CC080 (ISRRAI-172), RS3DC040 (ISRRAI-174) and RS3DC080 (ISRRAI-175).

Problem

Entry level server systems based on the S1200BTL (ATX) or S1200BTS (uATX) system boards can loose second LAN port functionality. The LAN port is not detected in the operating system; it is not available in device manager.

Products

• Intel S1200BTS based systems; operating system independant
• Intel S1200BTL based systems; operating system independant

Issue

This issue should only occur if you have recently completed one of the following:

• Completed a BIOS update
• Upgraded the processor (which can cause a BIOS reset)
• Reset the BIOS to defaults
• Had a motherboard replacement

In these instances the second LAN controller can be disabled due to a BIOS problem. You will need to manually re-enable the second LAN controller.

Root Cause

The root cause is a BIOS glitch that Intel have documented. Their resolution is described below.

Resolution

Ensure both controllers are enabled in the BIOS:

• From power-on, tap the F2 key to go into the BIOS
• Go across to the Advanced menu
• Go down to the PCI Configuration menu
• Ensure that Onboard NIC1 and Onboard NIC2 are enabled

Applies to:

• Server products based on the S1200BTS and S1200BTL motherboards.

Overview of the RAID Card Replacement Process

• If the system is still booting - ensure that the RAID drivers for the new card are installed, and/or a sufficient version to support the new card. For example, customers replacing an SRCSASRB or SRCSATAWB should be running driver version 4.x before trying to upgrade to the Intel RS2BL080.
• Remove old card; note the order of the drive cables and other signal cable connections.
• Check the old card for any Feature Keys which need to be moved to the new card.
• Move any RAID card battery to the new card if the old card had a battery unit fitted and the new card has not been supplied with a replacement, so long as the cards are the same model. If they are different models, a new battery will need to be requested.
• Disable Quiet boot in the BIOS.
• Fit the new card but leave the hard drive cables disconnected.
• Flash the latest approved firmware on the new card.
• Shut the system down; connect the drive cables.
• Boot the system up and follow the foreign configuration prompt to preview and then import the logical drives.
• Test the system. If a RAID driver installation or upgrade is needed and the system doesn't boot from the RAID (failing with a STOP 0x0000007b blue screen message) then a repair operating system installation may be needed.

Overview of the Motherboard Replacement Process

• Confirm the fault with the original motherboard, saving a copy of the hardware SEL LOG for reference.
• Replace the motherboard and refit all cards and components in the original slots. Ensure any RAID keys are transferred.
• Disable quiet boot. If you can't get into the BIOS to disable quiet boot, remove the RAID controller
• Upgrade the motherboard BIOS package (usually easiest done in the EFI Shell)
• Reboot the system after the upgrade and go back into the BIOS - its important the first POST after the BIOS upgrade is allowed to complete
• Ensure that Quiet boot is still disabled
• If the system is in a noise sensitive area - for example, the system is a workstation on a desk - set the system fan/performance mode to Acoustic
• If you removed the RAID controller earlier shut the system down, refit the controller and then go back into the BIOS
• Check the on-board SATA controller is in the correct mode (for example, IDE, RAID, AHCI - and sometimes with the RAID modes there may be multiple options)
• Make sure the BIOS boot order is correct
• Make sure the system warning LED is solid green and use the SEL LOG viewer to troubleshoot
• Test the system and retain a final SEL log for reference purposes

Applies to:

• Systems with an Intel or LSI Hardware RAID Controller or Module.
• Systems based on an Intel server platform

Problem

When you try and update the BIOS in Windows using the Intel OFU program ("One Boot Flash Update") tool, you get an error message saying that there was an "Error while parsing the cfg file".

Cause

This issue is caused by an out of date IPMB driver being installed in Windows. Typically an out of date IPMB driver might be in use if an older SELVIEWER, SYSINFO or Intel Active System Console has been installed or used previously.

Resolution

The Intel OFU BIOS Update utility can only run when an up to date IPMI/IPMB driver is installed. The OFU package contains an updated IPMI driver.

Instructions

• Locate the Driver folder in the OFU package
• Go to the x64 or x86 subfolder, depending on your version of Windows. For Windows Server, choose x64.
• Right hand click on the driver "install" file and use "Run as Administrator" to install it.

• Re-attempt the BIOS flash process.

• Reboot the system to complete the flashing process.
• Remember, that during the first reboot the system may need some time to complete the backup BIOS flash. This may add an extra 5 minutes to the POST time - do not reset or reboot the system in this time.

Applies to:

• Stone Server products supported by the Intel OFU Windows BIOS Flash Utility

Why Use RAID

RAID systems can add resilience against disk failures and/or combine the performance of many disks into a fast single volume. It is important that any faults or error messages with RAID systems are promptly examined to ensure that data is protected.

How to Monitor the RAID

You should always keep installed the RAID monitoring software for your RAID adapter. This includes:

• The LSI MegaRAID storage console (MSM) for LSI Hardware RAID Adapters.
• The Intel Raid Web Console 2 (RWC2) for Intel Hardware RAID Adapters or modules.
• The Intel Matrix Storage Console for SATA/AHCI controller based software RAID systems.

Hardware RAID is recommended for most server applications and this article deals with the management of an Intel Hardware RAID system.

Dos and Don'ts

• Always ensure that the proper RAID driver is installed and running. Do not rely on Microsoft provided RAID drivers. Driver version 6.x is recommended for Intel or LSI Hardware RAID Adapters.
• Always ensure that the latest RWC2 or MSM is installed. Minimum versions below:
  • For Intel adapters SRCSASRB, SRCSATAWB or RS2BL080, RWC2 version 12 or later should be installed.
  • For second generation 6G adapters such as RMS25CB080, RWC2 version 14 or later should be installed.
  • For first generation 12G adapters such as RS3DC080, RWC2 version 15 or later should be installed.
• Download MSM from the LSI/Avago web site and Intel RWC2 from the Intel site.
• If your system is old and being re-purposed or taken offline for maintenance, consider upgrading the firmware on your RAID controller. Ensure that the drivers are up to date and that you have a full, tested, disaster recovery backup first.
• Don't re-use desktop harddrives to replace server harddrives
• If a harddrive fails, replace the hard drive. Do not attempt to re-use a failed hard drive.

​

Planning your RAID System

When planning and implementing your storage system, consider the performance, capacity and resilience that your system requires.

For example, a VMWare virtual server in a SAN environment may be able to run the operating system using RAID1, as the operating system disks have little load most of the time (unless the system has more virtual RAM committed to guest operating systems than physical RAM available, in which case VMWare may page to disk and this will put a load on the local storage).

Another example is a file server with a large RAID array. In this situation, consider using RAID6 and possibly include a hot-spare as well.

Monitoring

Especially when the server is not in a location where you can hear audible failure messages, it is important to set up email alerting. Even if the server is is in a place where you would normally hear an alarm, the use of email alerts can help you be aware of problems when you are away from site.

Use the MSM / RWC2 Email Notification Feature

Steps to Setup Monitoring

• Login to RWC2 / MSM (use your Windows Administrator or Domain Administrator password to do this)

• Then, under Tools, click on Monitor Configure Alerts

• Use the Alert tab to confirm which type of event gets notified using the right delivery methods. For example, you do not want Information events delivered by email.

• Then use the Mail Server tab to configure the SMTP server settings. Example below.

• Finally, use the Email tab to list the recipients that should receive the email notifications.

Finally

Please review our other RAID articles as these cover topics such as troubleshooting, the RAID levels available, and the differences between desktop and enterprise class hard drives.

Applies to:

• All products capable of RAID functionality

BIOS Simulator

Please find below a link to the Intel BIOS simulator for server systems. This is an updated similator for the Xeon Processor E5-2600 v3 Product Family, including S2600WTT, S2600CP2 and S2600CP4.

This utility simulates the BIOS interface a user would normally see when pressing <F2> during POST. It can be used for support or research purposes.

Downloads

• Intel
• Alternative (latest as of 23/2/2016)

Required Component for the Simluator To Run: Microsoft Visual Basic PowerPacks 10.0

• Microsoft
• Alternative

Differences Between the Simulator and the Product

Whilst the simulator shows you the common options available on the S2600 v3 platform, there are some differences in the latest versions of the BIOS. The S2600WT example is shown below.

BIOS Screenshots

The screenshots below are taken from the S2600WT but broadly apply to the S2600WTT, S2600CP2, S2600CP4 and S2600CW2.

Pressing F2 on POST Brings you to a Pre-Setup Screen

Inside the Setup Menu, Most BIOS tabs are now listed top down, instead of across the top

Start > MAIN

Start > Advanced

Start > Advanced > Processor Configuration

Start > Advanced > Memory Configuration

Start > Advanced > Mass Storage Controller Configuration

Start > Advanced > Mass Storage Controller Configuration > SATA Port 0-5

Start > Advanced > Mass Storage Controller Configuration > sSATA Port 0-3

Start > Advanced > PCI Configuration

Start > Advanced > PCI Configuration > Processor PCIe Link Speed

Start > Advanced > PCI Configuration > Processor PCIe Link Speed > Socket 1

Start > Advanced > PCI Configuration > Processor PCIe Link Speed > Socket 2 [Where fitted]

Start > Advanced > System Acoustic and Performance Configuration

Start > Server Management

Start > Error Manager

Start > Boot Manager

Start > Boot Maintenance Manager

Start > Boot Maintenance Manager > Advanced Boot Options

Start > Boot Maintenance Manager > Legacy Hard Disk Order

Start > Boot Maintenance Manager > Change Boot Order

Start > Boot Maintenance Manager > Change Boot Order > (Item Selected)

Start > Save & Exit

Applies to:

• Stone S2600 server system families that include the Xeon E5 2600 v3 Processor.

The Intel S5000 based servers and workstations do not have an EFI based SELVIEWER tool. If Windows is not running, you need to use the DOS based tool in order to gather the hardware System Event Log (SEL).

1. Download the correct tool - here or here - for your S5000PSL, S5000PAL, or S5000XVN system.

2. Format a USB pen with the FAT32 file system. Make it bootable. To do this you will need a copy of the Windows 98SE boot files and a copy of the USB Format tool here or here (3rd party site).

3. Extract the DOS Selviewer download to the pen drive.

4. Boot the server from the pen drive. If necessary, go into the BIOS using the F2 key, and use the Boot Manager screen to directly boot from the pen drive.

5. When you have the DOS prompt, either run SELVIEW to lauch the GUI (which will allow you to save entries) or type in SELVIEW sn.sel /save to save a copy of the log entries directly to a file.

6. Send a copy of the file to Stone support for analysis.

Applies to:

• Intel S5000 Based platforms (which do not have an EFI SELVIEWER available)

Recovering a Failed LSI or Intel RAID Array

This article will show you how to recover a failed RAID 5 array on an Intel or LSI hardware RAID controller. The same or similar process can also be used with other RAID arrays such as RAID 1 or RAID 6.

It is important to understand first how drive configuration information is stored, and what a "foreign configuration" is, as you may see messages about this appear.

What is a Foreign Configuration?

Intel or LSI Hardware RAID controllers store RAID volume configuration information on both the drives and the controller. When the system is booted or a drive is inserted, the drives and configurations are examined and any configurations that don't match each other are flagged as a foreign configuration. The RAID controller will use its own configuration as the master record, to identify which configurations are invalid.

All physical drives configuration information contains a list of the drives involved in the associated logical drive.

What's the difference between Optimal, Degraded, Critical and Offline?

RAID controllers use these terms to show the health of the logical drive.

• Optimal: Everything should be normal; all drives are present
• Degraded: A failure has occurred; likely a drive is missing. However, the RAID array may be able to tolerate further failures. The bad drive should still be replaced as soon as possible.
• Critical: A failure has occurred and the RAID array health is critical. Any further failures will lead to the loss of data.
• Offline: The virtual drive has failed and the data will be inaccessible. Recovery can be attempted but is not guaranteed.

For example, RAID6 arrays can lose up to two drives. The loss of one drive will result in the logical drive being reported as degraded, not critical.

Scenario 1 - The system is booted but not all of the drives are plugged in, possibly due to a loose cable.

Messages such as the one above require prompt action:

• Switch off the server and check all the cables and connections, especially the seating of any multi-lan SFF-8087 cables. Reseat all drives and re-test.
• Do not accept the message above or load the configuration utility: Doing so will write a configuration message to the remaining drives detailing the drives which are missing.

​Scenario 2 - A drive has failed, but now a message about a Foreign Configuration has appeared

Hard drives which encounter problems such as bad blocks or a controller fault may stop being able to be detected. The drive may then become "available" again, perhaps after the drive has reallocated the bad blocks. However, when the drive failed, the RAID card will have written a configuration update to itself and the remaining drives, notifying the failed drive. The configuration on the failed drive is now out of date and does not match the other drives, so it is flagged as having a foreign configuration.

• If the RAID array is degraded or critical, it is usually safer to attempt to rebuild onto a fresh drive, rather than trying to either re-use a problem drive (i.e. rebuilding onto it), or worse, trying to force the previously failed drive online. This is because any disk writes that have occurred to the RAID pack - even just the normal day to day running of an idle Windows system - will mean that the drive that failed and has now re-appeared contains out of date data. Attempting to put this drive into use without rebuilding will result in serious system corruption.
• If the RAID array is marked as failed, always try and import the foreign configuration, making sure that the drive that failed last is the drive that you try and re-use. Drives that failed earlier (as perhaps you could have in a RAID 5 or RAID 6 situation) will not only have an out of date or foreign configuration, but more importantly the data on the drive will likely be out of date if an operating system has been booted or running since the failure.

​Scenario 3 covers importing a foreign configuration.

​Scenario 3 - The drive was disconnected accidentally, my array is offline and I need to recover it.

Disconnected drives will either be marked as Foreign, or Unconfigured Bad. Unconfigured bad drives cannot be have the foreign configuration imported until you mark the drive as "Unconfigured good".

Foreign drives detected on the RAID card boot may be flagged as below.

​

In this instance, either Press F, or better still, press CTRL+G to enter the RAID BIOS and manage the foreign configuration process.

Some older RAID cards may allow you to Import a foreign configuration without Previewing it. However we recommend you always preview the configuration to be imported and check that all logical drives - including those that have no problem - are correctly shown.

If you have different drives with different foreign configurations then you may have multiple choices to try and import. Try "All Configurations" first. If this does not work, try each configuration in turn, making sure you preview it first. It is likely that only the configuration from the last drive that failed will be importable.

Scenario 4 - I'm Running RAID 5 and my system crashed. I appear to have had two hard drives fail.

This can happen especially if you have a hot-spare drive. When the first drive fails, the RAID card will start rebuilding onto the spare. The unusual workload may cause another weak drive to then experience a failure.

In this scenario you may not may not get messages about a foreign configuration. If you do not, or if the foreign configuration does not import, follow the steps below:

• Enter the RAID BIOS console, and check for the presence of any drives marked as unconfigured bad.

• Click on the drive, and then choose the option to "Mark as Unconfigured Good", then click on Go.
• The RAID card likely will NOT prompt to import the foreign configuration at this time. You will need to reboot and then go back into the RAID BIOS to follow the foreign configuration import process.

In All Situations

• Always ensure that the RAID array is rebuilt when necessary. You can check the progress from the RAID BIOS or from the Intel RAID Web Console in Windows.

​

• Never disable the audible alarm. Always silence the alarm, even if you need to do so multiple times.
• Always contact Stone support if a drive is marked as failed, or having bad blocks/media errors. Drives in this situation should be replaced.
• When dealing with RAID problems, check the Media Error count or Predictive failure count to see if you are managing multiple problems.

​

What Can I do to Minimise the Chances of Unexpected Failures?

• Always have the correct version of RAID Web Console installed if you are running Windows, or use a Remote RAID Console plus the Management component if running VMWare ESXi.
• Set up RAID Web Console for email alerting of problems. This may alert you to minor problems or growing issues before a drive fails completely, for example.
• Take advantage of the option for a scheduled consistency check available in RAID Web Console for Window. Right hand click on the controller and then select "Schedule Consistency Check".

• To ensure your system can manage drive problems more reliable, always use a recent RAID driver, RAID firmware and RAID Web Console software when commissioning your system. Contact Stone support for further assistance.

Applies to:

• Server and Workstation products with LSI or Intel Hardware RAID Controllers up to and including the 2.5Generation 6G SAS Adapters (excludes the third generation adapters).

The EFI Shell

The Extensible Firmware Interface (EFI) shell is effectively a small operating system built into the system motherboard on modern servers. This provides an environment for troubleshooting and maintenance outside of the operating system.

This article describes how to access the EFI shell on an Intel Server or Workstation board and how to access resources on a USB pen drive.

How to Access Tools through the EFI Shell

Generic instructions are provided below. Exact screens and steps may vary from model to model.

• Prepare your USB key - it should be formatted using FAT32 and anything you wish to use should be pre-extracted from any ZIP file. Always extract updates or tools into simply named subfolders. The use of folders prevents the system from automatically running the update (by calling startup.nsh) when you enter the shell.
• Insert your USB key containing your tools or utilities into the server.
• Turn the system on
• Tap the F2 during POST to gain access to the main BIOS setup utility
• Go to the Boot Manager Menu and then select Internal EFI Shell. Press Enter to boot the EFI Shell

• Once you have entered the EFI shell you should get a screen similar to the one below. Your USB pen drive should be detected as fs0: and should be in the list of devices.

• If fs0: is not listed you will need to try and map the drive in by using the map -r command:

• If fs0: is still not listed, your pen drive may be faulty, you may need to try an alternative USB port, or you may even need to try a different model of pen drive.

• Now select the USB pen drive by typing in fs0: and pressing Enter.
• You should now be able to list the contents of the pen drive by using the DIR command:

• Notice that folders are listed in blue and EFI executables (.EFI) are listed in green
• Some utilities also have .NSH files - these are scripts. For example, most Intel update utilities have a startup.nsh file which can be automatically executed on entering the shell.

• Use the CD command as you would in DOS, to change directories. Its important to remember that you must put a space after the CD command.

A seperate article wil cover using the SELVIEW utility in both EFI and Windows environments.

Applies to:

• All Stone / Intel based server systems.

Problem

Customers using the AXXRMM4 (RMM4) or AXXRMM3 (RMM3) remote management modules may experience a security message when they try and use the remote keyboard/video/mouse (KVM) feature.

The security message may either completely block the page or warn that the application may be blocked in future, depending on the version of Java installed or the Java security settings in force.

Cause

This issue is caused by the Baseboard Management Controller (BMC) on the motherboard having KVM applet code which does not include the proper publisher identifier information.

Resolution

Work Around 1

• Tick the "I accept the risk and I want to run this application" box, and click Run, if your version of Java gives you this ability

Work Around 2

• Close the security prompt and browser Window.
• Open "Configure Java" in Windows.
• Go to the security tab and set the Security Level to Medium.
• Re-try launching the KVM from the browser.
• If this does not help, try "Restore Security Prompts" and re-open the KVM again.

Work Around 3

• In the Java Control Panel (as above), add the IP address of the RMM module to the Exception Site List by clicking on the Edit Site List button.

Resolution 1 (for W2600/S2600/S2400 series based systems only - Xeon E5)

• If the workarounds are not sufficient, upgrade the system firmware package with the May 2014 version or newer. This includes update BMC and RMM code to correct the publisher attribute.

Resolution 2 (for S5520HC and S5520SC only)

• If the workarounds are not sufficient, contact Stone Support for beta BMC 62 code which resolves this issue.

Applies to:

• All Stone Server products capable of using Remote Management Module 3 (RMM3) or Remote Management Module 4 (RMM4)

Problem

The Intel X710 Series 10Gbe Dual or Quad port network adapters may report that one or more ports cannot start. The problem may appear if the driver was recently upgraded either through Windows Update, or by downloading the updated driver from the Intel web site.

The network card instance in Device Manager is shown with an exclamation mark, indicating a problem.

Cause

The problem is due to an issue with the non-volatile memory area on the adapter

Resolution

• Upgrade the NVM area using the Intel Non-Volatile Memory (NVM) Update Utility for Intel® Ethernet Converged Network Adapters, here.
• This needs to be done inside the UEFI environment. Run the following command line inside the UEFI shell:

nvmupdate64e -u -l -o update.xml -b -c nvmupdate.cfg

• Then shut down the server, wait 30 seconds,and then power it up again
• Confirm that the issue is resolved - all Intel X710 adapters in device manager should show without any exclamation marks.

More Information

• This problem does not appear to happen if you use driver versins 22.01 or 21.1
• Driver version 22.3.108 or later appears to have this issue

Applies to:

• Stone servers or workstation with the Intel X710-DA2 (ISRNET-135) or X710-DA4 (ISRNET-136) network adapters