Stone Equinox / Storage Spaces Direct (S2D) Hyper-V Cluster UPS Shutdown Script

Storage Spaces Direct (S2D) nodes provide disk resiliency to each other as part of the storage network.

When an extended power outage occurs that exceeds the available battery runtime, it is important that a managed shutdown process is used:

• To ensure that all virtual machines are saved or shutdown
• To ensure that nodes do not start inadvertently start moving virtual machine loads between nodes
• To ensure that any individual node does not switch off until all cluster workload has ceased - this ensures that the S2D disk storage is consistent and not damaged

A software package is now available to provide this shutdown facility. It is compatible with all UPS software that supports shutdown via a script.

Latest version: v119 with Installer version v1.0.25 (09/01/2025)

See:

• Script Installation Process
  • Pre-Requisites
  • Installation on each S2D Host
  • Verifying Installation
• Configuring the UPS Software
• Testing the System
• Recommended UPS Hardware Installation
• Shutdown Settings

Script Installation Process

Pre-Requisites

A. Install your UPS Software as normal

Its a good idea to install your UPS Software first, as depending on the type of software installed, the software will be configured for you by the installation script.

B. Manual creation of a domain service account. Optional: If you do not follow this step to create a S2D Management domain account called svcUPs, the installation script will do it for you.

• In your S2D Management domain, create a domain service account called svcUPS
  • This account should have domain administrator rights, as it needs to be able to control the cluster
    • Either copy the existing domain administrator account, or make a fresh, new account, and add it the Domain Admins group
  • Make sure the account is set so that the password never expires
  • Leave the account object in the Users container, or somewhere else unaffected by group policy
  • Set a secure password on the account and make a note of it

Installation on each S2D Host

• Download the script and extract it to a suitable folder, for example C:\_STONE\S2DUPSShutdown
  • Do not extract to a network location
• You should have 3 files in the folder
  • Install_S2D_UPS_Shutdown_Script_(S2DUSS).ps1
  • Instructions.TXT
• Double click on Install_S2D_UPS_Shutdown_Script_(S2DUSS).ps1. This starts the installation and may prompt for administrator approval
• If you have already created the svcUPS account, enter in the svcUPS password when prompted

• Alternatively, if the account does not exist, you will be offered to create a new account with a new complex password
• Type in the new password. The account will be created in the S2D Management Domain Active Directory

• When you have supplied the password for the existing or new account, the script will complete the tool installation.
• When it completes, note the exact name and location of the CMD file that the UPS software must call.
  • Copy the CMD filename into your clipboard to make it easier to use later
  • Example: C:\Program Files\Stone\HAS\S2D UPS Shutdown Script\StartShutdown.CMD

• The installer will have renamed or created additional files:
  • StartShutdown.CMD (newly created) - this is the file that the UPS software must run
  • ShutdownControl.CMD - this does the heavy lifting of the shutdown procedure
  • And a Logs directory, for debugging shutdown events
• Remember to complete the script installation on all Nodes in the cluster, as well as UPS software installation and configuration

Verifying Installation

Check for a scheduled Task called UPShutdown in the \MANAGEMENT folder, in Task Scheduler

Configuring the UPS Software

Automatic Configuration

You should install your UPS software before running the UPS Shutdown script installer, as the script installer may be able to configure your software for you.

After installation, we recommend that manually check your UPS Software shutdown settings

Software Supported for Automatic Configuration

• Eaton Intelligent Power Protector
• Eaton Intelligent Power Manager
• Riello Powershield 3
• APC Powerchute Network Shutdown
• HPE Power Protector

Manual Configuration / Verifying your UPS Software Shutdown settings

• Then, under the shutdown configuration, select that a script is used.
• You must specify that a shutdown duration of 600 seconds (10 minutes) is used. This lets the UPS software allow plenty of UPS battery runtime to ensure the systems shutdown cleanly.

Testing the System

You should arrange full systems downtime to test the shutdown sequence. You can then test the software either from the UPS software, or by running StartShutdown.CMD from an Administrative command prompt

• Check the Logs directory later for Cluster and Host shutdown events
• You can also check the Application log under Stone S2D UPS Shutdown Script

Recommended UPS Hardware Installation

• When using Intel based servers, do not install UPS in parallel. For example, if you have two UPS and two nodes, dedicate one UPS to one Node
  • This is because of the active/passive nature of the system power supplies
  • In the event of one UPS battery lasting significantly longer than others, the Cluster (and all thus VMs) will still be shutdown by the first node triggered for shutdown
• Always ensure your network switches between the Hosts and the UPS Network Management card are also UPS protected
• Always periodically test your batteries using the run-time calibration feature
• Batteries that are 3 years old or more may require replacement

Shutdown Settings

All of the settings in S2D_UPS_Settings.ini are shown below.

Concepts:

• LocalVMs: Virtual machines running on the local node. These may or may not be clustered. Clustered VMs are shutdown first
• Save or shutdown: By default, Local VMs will be Saved unless they have a memory demand of more than 8192MB, in which case they will be shutdown
• The starting of shutting down or saving of local VMs will be staggered over 2 minutes
• When the script starts, it sets up a separate cluster shutdown job that will start after 2 minutes, during which time the local virtual machines will be saved or shutdown
• During this time, in most situations, the same script running on other nodes will be saving or shutting down VMs local to those other nodes
• By default, after 4 minutes of starting the cluster shutdown job (total 6 minutes), the script will see if it is safe to turn off the node:
  • When all local VMs (clustered and non-clustered) have completed shutdown/saving
  • When the cluster has shutdown
• The remaining 4 minutes of the shutdown duration are to ensure the cluster and then the host have enough time to complete shutdown

[Locations]
LogFilePath=C:\Program Files\Stone\HAS\S2D UPS Shutdown Script\Logs

[LocalVMSettings]
;maximum number of seconds to start shutting down all local VMs
MaximumTimetoStartShuttingdownLocalVMs=120
;maximum number of seconds in between starting each VM shutdown. Will be automatically reduced to shutdown within MaximumTimetoStartShuttingdownLocalVMs
LocalVMsStaggeredDelay=5
;Set this to False to Always Shut down local VMs, to Set to True to save VMs instead of shutting down, dependant upon each VMs memory demand and the OnlySaveLocalVMsBelowMemoryDemandThreshold setting
SaveLocalVMs=True
;Set this to False to Always Save local VMs, or set to True to only Save VMs that have a memory demand below the threshold. If set to True and the VM has a memory demand above the threshold, it will be shutdown instead of being saved
OnlySaveLocalVMsBelowMemoryDemandThreshold=True
SaveLocalVMsMemoryDemandThresholdMB=8192
;VMs with a memory demand above this threshold will be shutdown, instead of being saved

[ClusterSettings]
ShutdownCluster=True
OnlyInitiateClusterShutdownIfPoolOwner=False
;The amount of time in seconds before shutdown-cluster is issued
StartClusterShutdownDelay=120

[LocalNodePowerOff]
;Shutdown of this Node - in S2D, this needs to happen after all Cluster IO has ceased
TurnOffThisNode=True
;The settings below is the time in seconds, after Stage 2 has been started, that we will wait for the cluster shutdown job to complete before testing to see if we are ready to turn off the node
ClusterShutdownTimeout=240
;The settings below get the script to check every 5 seconds to see if these conditions have been met
DontTurnOffNodeifLocalClusteredVMsStillRunning=True
DontTurnOffNodeifLocalNonClusteredVMsStillRunning=True
DontTurnOffNodeifClusterIsStillRunning=True

[Misc]
Version=118

;Total Shutdown time should be:
;The greater of:
;MaximumTimetoStartShuttingdownLocalVMs + time to save or shutdown last VMs
;StartClusterShutdownDelay + ClusterShutdownTimeout
;Then a final optional checks for LocalClusteredVMs, LocalNon-ClusteredVMs, and then the Cluster itself
;Then issuing Local Host Shutdown
 

Applies to:

• Stone Computers Equinox
• Stone Computers Elysium
• Most S2D Hyperconverged Platforms

Booting from and Using UEFI / GPT Partitions

If your RAID or disk volume is larger than 2TB, you will need to use one of the following methods to use all of the capacity and to be able to boot from the volume:

• Use a standard MBR partitioning method, but create separate smaller volumes inside the RAID. Intel / LSI RAID cards allow you to create a 3TB RAID6 drive group, for example, but then create smaller volumes inside this, for example 500GB and 2.5TB. You would need to use a GPT volume for the 2.5TB volume to use all of the capacity, but could easily boot from the 500GB volume as this can be setup as a traditional MBR (master boot record) volume.
• Use a GPT partition. You will need an UEFI compatible BIOS and a GPT boot compatible operating system, which is normally recommended as being Windows 7 x64 or Server 2008R2 or later.

For more information, please see attached the Intel GPT white paper, available here, or attached.

Applies to:

• Server or Workstation systems with a hard drive or RAID volume larger than 2TB.

Driver Package

The latest Stone Equinox Driver Package for Windows Server 2019 is below

• December 2021 Pack

Applies to:

• Stone Equinox

Problem

Newer versions of the Realtek LAN Driver, from around mid-2014 onwards, may fail to import into Windows Deployment Services, either WDS 2008R2 or WDS 2012.

WDS reports that the package addition failed. However, no actual error code or meaningful reason for the failure is displayed.

Cause

The root cause appears to be a problem with the WDS Jet Database and either the size of the Realtek LAN driver, or some particular information inside the driver INF. If you use the WDS Powershell utilities to try and import the driver an error code is return which indicates that a database error is being generated, with a record being "too big".

Resolution

Server 2012

There is one full resolution for this issue and three alternatives should your situation require it. However with the release of the April 2016 Realtek driver, only the full resolution should be required.

• Obtain the April 2016 Realtek drivers or newer and use the drivers in the WinPE or WDS sub-folder. These drivers are also now available in our SCCM / WDS packs.

Server 2008R2

As of April 2017, some x64 versions of the Realtek driver, even the WinPE version, do not import into WDS for Server 2008R2. There is no resolution for this apart from upgrading your Server infrastructure or the work-arounds below.

Other Work-Arounds

1. If using MDT, import the driver directly into MDT and then rebuild your Lite-Touch image.
2. Alternatively, if supported by the hardware you have, use an older version of the driver, such as this one from August 2013 for Windows 7.
3. Or, manually add the driver to the boot WIM file manually for situations where you are capturing or deploying images.
4. Finally, for network based installs, whilst you can add the driver to the boot WIM file, you could to manually deploy the Realtek LAN driver to the new installation. Note however that joining the machine to the network may not be possible until this has been done. This method is not suited to large deployments.

Overview of Adding a Driver to a WIM file

1. Download and Install the latest Windows Assessment and Deployment Kit (ADK) to your PC. This can be installed on the WDS server however its not normally recommended, especially if the server is a domain controller. As of 13/4/17, the latest ADK is 1703 for Windows 10 (the Windows 8.1 Update ADK is here).
2. Copy the boot WIM file to your ADK PC. Always keep a backup copy of the original WIM file.
3. From the Deployment Tools command prompt, mount the WIM file to a folder. The destination folder must exist, and should be empty:

Imagex /mountrw "d:\wimwork\boot.wim" 1 d:\output

4. Now add all of the drivers that you want. Put all of the drivers in a Drivers folder as below:

Dism /Image:d:\output /Add-Driver /Driver:d:\drivers /Recurse /ForceUnsigned

5. (The /ForceUnsigned switch will allow all drivers to be added, whether signed or not).
6. Now commit the changes back to the boot WIM file:

Imagex /unmount /commit d:\output

7. Now you can copy the WIM file (d:\wimwork\boot.wim in the example) back to the WDS server and test to see if the driver addition was successful.

Applies to:

• All Stone desktop and laptop products with Realtek LAN adapters.

SEL Troubleshooting Guide

Intel produce the SEL guide to help you understand the contents of the SEL and SYSINFO Logs.

For example, translate CPU numbering used in the logs to the numbering used in the motherboard manual or TPS

Applies to:

• Stone / Intel Xeon Servers and Workstations based on the Intel® Server Board S2600/S7200/S4600/S2400/S1600/S1400 Product Families

Intel RAID Manual

Intel have produced a manual which covers the setting up and basic maintenance of their RAID systems, primarily those based around the LSI RAID stack. This includes controllers such as the RMS25CB080 and RS2BL080.

The manual can be downloaded here, or attached.

Applies to:

• Intel / LSI Hardware and Firmware RAID Products

Driver Package

The latest Stone Eclipse Driver Package for Windows Server 2019 is below

• May 2021 Pack

Applies to:

• Stone Eclipse

Technical Product Specification Documents

Intel produce these TPS documents for each family of server or workstation motherboard. Use these documents to:

• Identify supported families or versions of processors.
• Find out supported memory configurations and DIMM population instructions.
• Determine the controller or type of SATA ports available.
• Find out the configuration of PCI Express Slots.
• Get details about all connections, headers and jumpers.
• Locate and use diagnostic LEDs.

​

Intel Server Systems

Intel Server Systems include Intel server or workstation motherboards. The second part of the Intel Server System model code indicates which motherboard is fitted. For example, the R2000GZ series of Intel Server system indicates that it includes an S2600GZ motherboard. The R2000GZ series includes different 2U models with different drive bay configurations.

Spares / Accessories List and Configuration Guide for Server Systems

These guides are available for Intel Server systems to allow research into the spare parts and accessories that are available for that model family.

S2600GZ / GL - R1000GZ/GL Server System and R2000GZ/GL Server System - Spares / Accessories List and Configuration Guide

S2600WT - R1000WT Server System and R2000WT Server System - Spares / Accessories List and Configuration Guide

Applies to:

• Stone / Intel Xeon Servers and Workstations

Azure Stack HCI Trusted Enterprise Virtualisation Solution Brief

Virtualization-based security (VBS) is a key component of the security investments in Microsoft Azure Stack HCI to protect hosts and virtual machines from security threats.

The attached solution brief illustrates how you can leverage VBS with Microsoft Azure Stack HCI and the Stone Computers Elysium SF (Hybrid Acceleration Series)

Applies to:

• Stone Elysium SF

Selected models of servers come with or can be upgraded to include a remote management module, such as the Intel RMM4. Use the instructions below to setup your remote management module.

Prerequisites

You will need the following:

• A static IP address for the management connection.
• The system being used for remote KVM control will need to have Java installed.

Recommendations:

• Use the RMM4 module option with a dedicated NIC management port where possible
• If you dont have the dedicated management NIC (RMM4 Lite) the motherboards first onboard port can be use for management connectivity. In this instance, we recommend against including the management enabled port in a 802.3ad static or dynamic trunk or team.

Instructions

• Boot into the BIOS using F2 from power-on.
• Go to the Server Management Menu
• Go to the BMC LAN Configuration Menu

• Configure the LAN settings as required. If you have the RMM4 module with the dedicated management port, it is recommended that you configure the Intel (R) RMM4 LAN Configuration settings only, and leave the BMC LAN Settings blank.
• The example below shows the RMM4 LAN settings configured as dynamic, with a host name configured to enable the device to be found easily on the network. However, static settings are recommended with a DNS Host entry configured on your DNS server.

• To enable the remote connection, a user account must be configured. We recommend you leave the default accounts and instead configure User3. Go down to User ID and select User3 from the list.

• Then set the privlege status - for example, to Administrator, set the user status to Enabled, and then set a username and password. The user name and password are both case sensitive.

Connection

To connect to the management console, use a web browser and connect to the IP address or hostname you specified.

For example:

http://172.16.5.182

or

http://dcmi

You will need to have Java installed to use the remote KVM (keyboard/video/mouse) and media redirection.

Getting More Help

For more information, see the Intel RMM4 user guide or contact Stone support.

Applies to:

• Server and workstation products equipped with an Intel RMM3 or RMM4 remote management module.