Enterprise Health Check

(This blog entry was in the publication queue for a while, so Chris Beal overtook me with his great blog entry)

For a very long time there are some security benchmarks in Solaris to assess your systems automatically with the compliance. Those benchmarks define a rich ruleset which is used to check your system if they adhere to this rules or not.

Best practices for a well-kept systems are often some really basic rules as well. For example „Do this“ or „Do not do this“. Thus when you already have a tool to check your systems based on rules for security, using it as well to check it for best-practice adherence is a quite obvious second step.

Oracle used this in the past to check your system for

root@solaris:~# pkg install ehc-solaris-policy@latest

Like the stackdb package used for the automatic analysis of core dumps for known issues, this package can be and must be independently updated. The idea is to give you the newest version of those package without the need to do the update.

root@solaris:~# pkg update ehc-solaris-policy@latest

When you want to run the benchmark to check for this rules, you just use the compliance tool.

root@solaris:~# compliance assess -b ehc
Assessment will be named 'ehc.2021-03-10,04:30'

Title
        Pool versions
Rule
        EHC-ZFS-00010
Result
        pass
[..]
Title
        Legacy Packages
Rule
        EHC-EOF-00010
Result
        fail

Currently (SRU 30) the following tests are implemented:

• EHC-EOF-00010 Legacy Packages: Legacy packages represent features which will be removed from future version of Oracle Solaris.
• EHC-ZFS-00120 Snapshot usage: ZFS snapshots are hidden from zfs list output generally, they can consume a large amount of storage which isn’t obvious. This test checks how much is in use by snapshots and reports if it is greater than 50%
• EHC-ZFS-00100 Slow I/Os: If we are getting slow I/Os to a disk it may result in poor performance.
• EHC-ZFS-00090 Last Scrub: Make sure the last scrub of all pools is within the last 90 days.
• EHC-ZFS-00070 Full pools:
  Report any pools over 90% full. This can lead to performance issues.
• EHC-ZFS-00050 Pools are not ONLINE: Check the status of pools and report any that are not ONLINE.
• EHC-ZFS-00040 Pools are made up of the same type of disk: Make sure pools contain the same type of disk.
• EHC-ZFS-00030 Disks in a pool all have the same firmware: Make sure disks in a pool all have the same firmware version.
• EHC-ZFS-00020 Mirrored rpool: Make sure the rpool contains a mirrored pair of disks. Keep in mind that, that the mechanism behind this can’t detect if you use an already mirroed device (for example an iSCSI/FC-LUN in a ZFSSA). However you could argue that a single device zpool is always a subclever idea with ZFS even when you have LUNs on a mirrored backend. But that is a somewhat religious discussion.
• EHC-ZFS-00010 Pool versions: Report any pools with old pool versions.

As this is a normal compliance benchmark you could automate the use of this benchmark on a multitude of system by using the compliance roster feature. I wrote a blog entry about this during the 11.4 beta phase.