Masa's blog

Fujitsu ServerView Agents and ServerView RAID Manager for Ubuntu

Fujitsu ServerView Agents is a server management agent for Fujitsu PRIMERGY servers, and Fujitsu ServerView RAID Manager is a RAID management utility for RAID controllers in PRIMERGY. The latest ServerView Agents and RAID Manager for Ubuntu published on Fujitsu download site are old, and does not support the latest platforms. I have ported SV Agents 7.20.28 for Linux and SV RAID Manager 6.4.3 for Linux to Ubuntu 14.04 and 16.04 and created deb packages to manage Fujitsu PRIMERGY servers running Ubuntu.

Here are the packages I ported.

serverview-raid-manager_6.4-3_amd64.deb
serverview-connectorservice_2.10.03-01_amd64.deb
serverview-eecd_7.20-28_amd64.deb
serverview-agents_7.20-28_amd64.deb
serverview-ssmwebui_7.20-28_all.deb

I am not authorized to distribute these packages, so if you need deb packages, please ask Fujitsu support. Fujitsu may have the latest packages. I just could not find them on the Fujitsu site.

Here are some screenshots.

ServerView System Monitor Web GUI

ServerView RAID Manager

ServerView Agents status check command output (srvmagt status)

# srvmagt status

Installation script for ServerView Agents Linux v7.20-28
(c) Fujitsu Technology Solutions 2016

Getting status of ServerView Agents Linux ...
               Server model: PRIMERGY BX2580 M2
                   Hostname: bx900s1-u-b8 (192.168.64.148)
           Operating system: Ubuntu 16.04 LTS [UBUNTU1604]
               Linux kernel: 4.4.0-28-generic on x86_64, SELinux unknown, EDAC enabled, target system
            ServerView eecd: serverview-eecd    7.20-28
  ServerView SNMP subagents: serverview-agents  7.20-28
      System Monitor Web UI: serverview-ssmwebui        7.20-28
ServerView Remote Connector: serverview-connectorservice        2.10.03-01
            ServerView Raid: serverview-raid-manager    6.4-3
               SNMP service: snmpd      5.7.3+dfsg-1ubuntu4

    ServerView module smbus: exists, loaded, /dev/pci/smbus exists
        Native IPMI modules: requested, loaded, /dev/ipmi0 exists

        SNMP daemon (snmpd): running (1629), UDP #161 OK, AgentX socket OK SNMP communication OK
     RAID daemon (amDaemon): running (1646), TCP 3173 OK
     ServerView eecd daemon: running (40798), /dev/eecd[as] OK
          SVRemoteConnector: running (17877), TCP 3172 OK
           ServerView scagt: running (42222), status OK (1)
          ServerView sc2agt: running (42234), status OK (1)
          ServerView busagt: running (42246), status OK (1)
           ServerView hdagt: running (42256), status OK (1)
         ServerView unixagt: running (42266), status OK (1)
        ServerView etheragt: running (42276), status OK (1)
         ServerView biosagt: running (42286), status OK (1)
        ServerView securagt: running (42298), status OK (1)
       ServerView statusagt: running (42308), status OK (1)
          ServerView invagt: running (42318), status OK (1)
          ServerView thragt: running (42329), status OK (1)
           ServerView vvagt: running (42340), status OK (1)
        ServerView hpsimagt: running (42350), status OK (1)
          ServerView vmeagt: running (42394), status OK (1)
     ServerView svupdateagt: running (42360), status OK (1)
           ServerView osagt: running (42370), status OK (1)
      ServerView vmesnmpagt: running (42417), status OK (1)
Getting status completed with result 0 (OK status).

vCenter Server Event Log is Filled Up with Log in/Log out Events from Fujitsu ServerView Operations Manager

If you monitor Fujitsu PRIMERGY servers running VMware ESXi with Fujitsu ServerView Operations Manager (SVOM), you probably notice that there are many log in/log out events from your SVOM server recorded in the event log (see the following screenshot for example):

By default, SVOM checks the host status through Fujitsu ServerView CIM provider which is included in Fujitsu Custom ESXi 5.x/6.0 images every five minutes (300 seconds). That is why you see the log in/log out events in every 5 minutes. In case you want to see some important events/messages in the event log like when a failure occurs, these unimportant login/logout events make you harder to find what you are looking for. You can change the monitoring interval or even disable the polling. To do this, follow the procedure described below.

Log in to your SVOM.
Open up a ServerList window.
Select the host where you want to change the interval, and bring up the Server Properties window.
Go to the Network/SNMP tab, and change the interval in Poll Interval. In this example, I changed it to 600 seconds (10 minutes). If you set this value to 0, that means the polling will be disabled for this host. Once you confirm the value, click the OK button to apply the change.

If you are using Fujitsu PRIMERGY servers and all are running ESXi, it would be better to use ServerView plugin for vCenter Server. With the plugin, you can monitor all your PRIMERGY server hardware status within vCenter Server. Also you can launch iRMC which is baseboard management controller on PRIMERGY (similar to DELL iDRAC, or HP iLO), open a remote KVM, and update BIOS and firmware.

You can download the latest plugin (as of 3/2/2016, it's V2.2.8) from the Fujitsu download site. Here is the direct link to the download page.

FUJITSU Software ServerView Plug-in for VMware vCenter V2.2.8

Fujitsu ServerView Operations Manager showing Link Up/Link Down alerts in every one or two minutes for Windows Server

When you manage your Fujitsu PRIMERGY servers running Windows Server with ServerView Agents, you sometimes see a lot of "Link Up/Link Down" alerts in ServerView Operations Manager like the following screenshot:

In this case, check what interface is reporting the link problem. In the above example, it shows "Communication link failed at the station 15.", so the interface index you should check is 15. In most cases, it is "Teredo Tunneling Interface".

Log in to the Windows Server system, and bring up a command prompt. Run the following commands to find out the interface:

C:\> netsh interface ipv4 show interfaces
C:\> netsh interface ipv6 show interfaces

In the above, the interface index 15 is "Teredo Tunneling Pseudo-Interface", so if you don't need this, disable it.

C:\> netsh interface teredo set state disable

This should stop the frequent up/down events in SVOM. If you see this for an interface other than "Teredo Tunneling Pseudo Interface", you probably have a real connection issue.

IPMITOOL on VMware ESXi

The IPMITOOL command is a great tool to configure the local BMC network, get the sensor/FRU information, etc. I use this command a lot on my Linux systems. Since virtualized servers (mostly VMware ESXi) are increasing, I want to use the ipmitool command in ESXi.
There are lots of blog posts talking about creating an ipmitool binary for ESXi out there. I would like to take a (little) further step to integrate the tool to my standard ESXi image so that I can deploy ESXi to my servers with the ipmitool command integrated.
Here is what I did.

Create an ipmitool binary. See the following blogs for details:
- ESXi ipmitool(works on any Linux system as well)
- Make Ipmitool Working For ESXi 5.1
Create an ipmitool vib. I used VIB Author on my Linux system (CentOS 6.7 x86_64). If needed, you can download the ipmitool vib I created from here (ipmitool-1.8.15-1.vib).
Create a custom ISO (and an offline bundle) with the ipmitool vib. Read the vSphere manuals on how to create a custom ISO/offline bundle using Image Builder. This VMware blog entry would help:
- Using the vSphere ESXi Image Builder CLI
Deploy the new image to servers.

I used the VIB I created in step 2 with ESXi 5.5 U3 and ESXi 6.0 U1, and it worked on both versions.

Here are some outputs.

~ # vmware -vl
VMware ESXi 5.5.0 build-3248547
VMware ESXi 5.5.0 Update 3
~ # esxcli software vib list |grep ipmitool
ipmitool                       1.8.15-1                               IPMItool  CommunitySupported  2016-01-14
~ # /opt/ipmitool/bin/ipmitool mc info
Device ID                 : 52
Device Revision           : 2
Firmware Revision         : 1.00
IPMI Version              : 2.0
Manufacturer ID           : 10368
Manufacturer Name         : Fujitsu Siemens
Product ID                : 1062 (0x0426)
Product Name              : Unknown (0x426)
Device Available          : yes
Provides Device SDRs      : no
Additional Device Support :
    Sensor Device
    SDR Repository Device
    SEL Device
    FRU Inventory Device
    IPMB Event Receiver
    IPMB Event Generator
    Chassis Device
Aux Firmware Rev Info     :
    0x07
    0x52
    0x00
    0x46
~ # /opt/ipmitool/bin/ipmitool lan print
Set in Progress         : Set Complete
Auth Type Support       : NONE MD2 MD5 PASSWORD OEM
Auth Type Enable        : Callback : MD5 PASSWORD OEM
                        : User     : MD5 PASSWORD OEM
                        : Operator : MD5 PASSWORD OEM
                        : Admin    : MD5 PASSWORD OEM
                        : OEM      : MD5 PASSWORD OEM
IP Address Source       : BIOS Assigned Address
IP Address              : 192.168.14.37
Subnet Mask             : 255.255.255.0
MAC Address             : 90:1b:0e:80:70:a9
SNMP Community String   : public
IP Header               : TTL=0x40 Flags=0x40 Precedence=0x00 TOS=0x10
BMC ARP Control         : ARP Responses Enabled, Gratuitous ARP Disabled
Gratituous ARP Intrvl   : 10.0 seconds
Default Gateway IP      : 192.168.14.1
Default Gateway MAC     : 00:00:0c:07:ac:40
Backup Gateway IP       : 0.0.0.0
Backup Gateway MAC      : 00:00:00:00:00:00
802.1q VLAN ID          : Disabled
802.1q VLAN Priority    : 0
RMCP+ Cipher Suites     : 0,1,2,3,6,7,8,11,12,15,16,17
Cipher Suite Priv Max   : XaaaaaaaaXXaXXX
                        :     X=Cipher Suite Unused
                        :     c=CALLBACK
                        :     u=USER
                        :     o=OPERATOR
                        :     a=ADMIN
                        :     O=OEM
~ # /opt/ipmitool/bin/ipmitool fru print 2
 Chassis Type          : Blade
 Chassis Part Number   : ABN:K1467-V200-54
 Chassis Serial        : YLSR000000
 Chassis Extra         : BX2580M1
 Chassis Extra         : FUJITSU
 Board Mfg Date        : Mon Dec  8 22:02:00 2014
 Board Mfg             : FUJITSU
 Board Product         : D3321
 Board Serial          : SQ1449MA00000
 Board Part Number     : A3C40161777
 Board Extra           : D3A
 Board Extra           : 02
 Board Extra           : BX2580 M1
 Product Manufacturer  : FUJITSU
 Product Name          : PRIMERGY BX2580 M1
 Product Part Number   : ABN:K1467-V200-54
 Product Version       : C
 Product Serial        : YLSR000000
 Product Asset Tag     : 02
 Product Extra         : a26799
 Product Extra         : 0426

Update on 02/26/2016: I changed the link to the VIB package. You should be able to download it without asking me for access permissions.

How to turn off the "Invalid or Missing Entitlement Certificates" message on RHEL

If your RHEL system is not registered with Red Hat Network, you will see the warning message, "Invalid or Missing Entitlement Certificates. This system is missing one or more valid entitlement certificates.", at the top right corner after you log in to the system at run level 5.

Of course, the message will be automatically turned off after you register your system with Red Hat. However if you just want to suppress the message without registering the system, that is possible with a few simple steps. What you need is to disable the rhsm-icon process. Here is the procedure.

Log in to your RHEL system.
Open a terminal, and change the current directory to /etc/xdg/autostart

$ cd /etc/xdg/autostart

Rename the rhsm-icon.desktop file.

$ mv rhsm-icon.desktop rhsm-icon.desktop.bak

Log out and log back in to the system

Now you don't see the warning message :)

Setting up JBOD with Fujitsu D2607 RAID Card on PRIMERGY Servers

Fujitsu does not officially support JBOD in Fujitsu 6Gb RAID 0/1 RAID card (D2607) on their PRIMERGY servers, and the JBOD mode is disabled by default. However you can enable it and use disks in JBOD with D2607.
Here is a procedure on how to enable JBOD in D2607.

Power on your PRIMERGY server.
Press the Control and Y keys during the RAID card initializaton (see below) to launch the command line interface.

Run the AdpGetProp command to check the current javascript:;setting.

Run the AdpSetProp command to enable JBOD if it is disabled.

Enter the 'q' key to exit the CLI.
Reboot.

If the drives which you want to use in JBOD are not shown as JBOD drives like "Unconfigured Good", you need to make the drives JBOD.

Please note that, as I said at the beginning, Fujitsu does not support the JBOD mode with the D2607 RAID card. You can enable and use it, but it should be done at your own risk. If you need D2607 JBOD support, please ask your Fujitsu contact for support.

Update on April 11th: I tested the above configuration on a previous generation of Fujitsu servers (S6 generation). I borrowed an S7 server (RX300 S7) with 16 HDDs and D2607, and did the same setting. The system hung when the RAID controller was being initialized.

When I used a later version of unsupported firmware, the system was able to boot with all the 16 drives in JBOD. I could install Solaris 10 and use all the drives in ZFS as JBOD. Therefore the issue would be caused by the Fujitsu firmware.

Bilding sysbench 0.4.12 on RHEL5

Here is what I got and what I did to compile sysbench 0.4.12 on my RHEL5 box. My system is running RHEL 5.4 x86_64 (kernel 2.6.18-164.el5).

The first step is to download the source code from the sysbench site, and run the normal commands as usual.

$ tar zxvf sysbench-0.4.12.tar.gz
$ cd tar zxvf sysbench-0.4.12
$ ./configure
$ make

Then I got the following error:

...
../libtool: line 838: X--tag=CC: command not found
../libtool: line 871: libtool: ignoring unknown tag : command not found
../libtool: line 838: X--mode=link: command not found
../libtool: line 1004: *** Warning: inferring the mode of operation is deprecated.: command not found
../libtool: line 1005: *** Future versions of Libtool will require --mode=MODE be specified.: command not found
../libtool: line 2231: X-g: command not found
../libtool: line 2231: X-O2: command not found
../libtool: line 2231: X-rdynamic: command not found
../libtool: line 1951: X-L/usr/lib64/mysql: No such file or directory
../libtool: line 1951: X-L/usr/lib64: No such file or directory
../libtool: line 2400: Xsysbench: command not found
...

As you see, there are many libtool errors, so I ran the autoreconf command to generate all build related files.

$ touch NEWS AUTHORS
$ autoreconf -ifv
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force
autoreconf: configure.ac: tracing
autoreconf: running: libtoolize --copy --force
Putting files in AC_CONFIG_AUX_DIR, `config'.
autoreconf: running: /usr/bin/autoconf --force
autoreconf: running: /usr/bin/autoheader --force
autoreconf: running: automake --add-missing --copy --force-missing
autoreconf: Leaving directory `.'

Ok. Let's re-run the commands.

$ ./configure
$ make

The errors are gone now. Next step is to install the compiled programs with the 'make install' command.

$ sudo make install

The last step is to run the sysbench command to make sure if it works ok.

$ sysbench --test=cpu run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 10000


Test execution summary:
    total time:                          9.5088s
    total number of events:              10000
    total time taken by event execution: 9.5048
    per-request statistics:
         min:                                  0.95ms
         avg:                                  0.95ms
         max:                                  1.20ms
         approx.  95 percentile:               0.95ms

Threads fairness:
    events (avg/stddev):           10000.0000/0.00
    execution time (avg/stddev):   9.5048/0.00