• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Ak12 pam
 

Ak12 pam

on

  • 326 views

 

Statistics

Views

Total Views
326
Views on SlideShare
326
Embed Views
0

Actions

Likes
0
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Ak12 pam Ak12 pam Document Transcript

    • =~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2012.11.13 17:12:00 =~=~=~=~=~=~=~=~=~=~=~=Boot Loader version 1.8Copyright (C) 2000-2003 Broadcom Corporation.Portions Copyright (C) 2002-2009 NetAppCPU Type: Dual-Core AMD Opteron(tm) Processor 2218Starting AUTOBOOT press Ctrl-C to abort...Loading x86_64/kernel/primary.krn:................0x200000/491239280x30d9258/27512016 0x4b15f28/7980237 0x52b23f5/3 Entry at 0x00202018Starting program at 0x00202018Press CTRL-C for special boot menuThe platform doesnt support service processorWed Nov 14 01:12:11 GMT [nvram.battery.state:info]: The NVRAM battery iscurrently OFF.Wed Nov 14 01:12:17 GMT [nvram.battery.turned.on:info]: The NVRAM battery isturned ON. It is turned OFF during system shutdown.Wed Nov 14 01:12:18 GMT [iomem.card.enable:info]: Caching module in slot 1 hasbeen enabled.Wed Nov 14 01:12:19 GMT [cf.nm.nicTransitionUp:info]: Interconnect link 0 is UPNetApp Release 7.3.6: Thu Jul 7 01:48:12 PDT 2011Copyright (c) 1992-2011 NetApp.Starting boot on Wed Nov 14 01:12:08 GMT 2012Wed Nov 14 01:12:27 GMT [diskown.isEnabled:info]: software ownership has beenenabled for this systemWed Nov 14 01:12:30 GMT [sas.link.error:error]: Could not recover link on SASadapter 3c after 10 seconds. Offlining the adapter.Wed Nov 14 01:12:30 GMT [sas.link.error:error]: Could not recover link on SASadapter 3b after 10 seconds. Offlining the adapter.Waiting for givebackReservation conflict found on this nodes disks!Local System ID: 151729197Press Ctrl-C for Maintenance menu to release disks.Wed Nov 14 01:12:30 GMT [ses.giveback.wait:info]: Enclosure Services will beunavailable while waiting for giveback.WDisk reservations have been releaseded Nov 14 01:12:35 GMT [ses.giveback.restartAfter:info]: Enclosure Servicesrestarting after release of reservations.Wed Nov 14 01:12:35 GMT [cf.nm.nicReset:warning]: Initiating soft reset onCluster Interconnect card 0 due to rendezvous resetWed Nov 14 01:12:35 GMT [cf.rv.notConnected:error]: Connection for cfo_rv failedWed Nov 14 01:12:35 GMT [cf.nm.nicTransitionDown:warning]: Cluster Interconnectlink 0 is DOWNWed Nov 14 01:12:35 GMT [cf.rv.notConnected:error]: Connection for cfo_rv failedWed Nov 14 01:12:37 GMT [cf.nm.nicTransitionUp:info]: Interconnect link 0 is UPWed Nov 14 01:12:37 GMT [fmmb.current.lock.disk:info]: Disk 0a.17 is a local HAmailbox disk.Wed Nov 14 01:12:37 GMT [fmmb.current.lock.disk:info]: Disk 0d.45 is a local HAmailbox disk.Wed Nov 14 01:12:37 GMT [fmmb.instStat.change:info]: normal mailbox instance onlocal side.Wed Nov 14 01:12:38 GMT [shelf.config.mpha:info]: All attached storage on thesystem is multi-pathed HA.Wed Nov 14 01:12:39 GMT [fmmb.current.lock.disk:info]: Disk 0b.16 is a partnerHA mailbox disk.Wed Nov 14 01:12:39 GMT [fmmb.current.lock.disk:info]: Disk 0c.17 is a partnerHA mailbox disk.
    • Wed Nov 14 01:12:39 GMT [fmmb.instStat.change:info]: normal mailbox instance onpartner side.Wed Nov 14 01:12:40 GMT [cf.fm.partner:info]: Cluster monitor: partnerakfiler13Wed Nov 14 01:12:40 GMT [cf.fm.timeMasterStatus:info]: Acting as cluster timeslaveWaiting for giveback...(Press Ctrl-C to abort wait)Waiting for giveback...(Press Ctrl-C to abort wait)Wed Nov 14 01:13:31 GMT[shelf.module.configuration.failure:error]: ERROR--Could not complete theconfiguration of disk shelf module B on channels 0b/0c disk shelf ID 1.Wed Nov 14 01:13:31 GMT [shelf.module.configuration.failure:error]: ERROR--Couldnot complete the configuration of disk shelf module B on channels 0b/0c diskshelf ID 2.Waiting for giveback...(Press Ctrl-C to abort wait)Wed Nov 14 01:15:29 GMT[sas.cable.degraded:warning]: Cable attached to SAS port "3d" is functioning ina degraded mode.Wed Nov 14 01:15:39 GMT [sas.link.error:error]: Could not recover link on SASadapter 3d after 10 seconds. Offlining the adapter.Wed Nov 14 01:15:53 GMT [shelf.config.tomixedha:info]: System has transitionedto a mixture of single and multi-path HA attached storageWed Nov 14 01:15:54 GMT [ses.multipath.ReqError:CRITICAL]: SAS-Shelf24 detectedwithout a multipath configuration.Wed Nov 14 01:16:10 GMT [sas.cable.degraded:warning]: Cable attached to SAS port"3a" is functioning in a degraded mode.Wed Nov 14 01:16:20 GMT [sas.link.error:error]: Could not recover link on SASadapter 3a after 10 seconds. Offlining the adapter.Wed Nov 14 01:16:20 GMT [disk.releaseFailed:error]: Disk release failed on3a.10.0 CDB 0x5f:0601 - SCSI:no sense (0 0 0)Wed Nov 14 01:16:21 GMT [ses.access.noMoreValidPaths:CRITICAL]: No more validpaths to Enclosure Services in shelf 11 on channel 3a.Wed Nov 14 01:16:21 GMT [ses.access.noMoreValidPaths:CRITICAL]: No more validpaths to Enclosure Services in shelf 10 on channel 3a.Wed Nov 14 01:16:21 GMT [ds.sas.element.xport.error:error]: Shelf module A XPORTERROR on channels 3a/PARTNER shelf id 11Wed Nov 14 01:16:21 GMT [ds.sas.element.xport.error:error]: Shelf module A XPORTERROR on channels 3a/PARTNER shelf id 10Wed Nov 14 01:16:21 GMT [raid.disk.missing:info]: Disk 3a.10.0 Shelf 10 Bay 0[NETAPP X412_S15K7560A15 NA00] S/N [6SL3SYV30000N2406M4C] is missing from thesystemWed Nov 14 01:16:21 GMT [raid.disk.missing:info]: Disk 3a.10.1 Shelf 10 Bay 1[NETAPP X412_S15K7560A15 NA00] S/N [6SL3S1HQ0000N239HAWN] is missing from thesystemWed Nov 14 01:16:21 GMT [raid.disk.missing:info]: Disk 3a.10.2 Shelf 10 Bay 2[NETAPP X412_S15K7560A15 NA00] S/N [6SL3NLK80000N239HAGX] is missing from thesystemWed Nov 14 01:16:21 GMT [raid.disk.missing:info]: Disk 3a.10.3 Shelf 10 Bay 3[NETAPP X412_S15K7560A15 NA00] S/N [6SL3PLKS0000N238HXQ8] is missing from thesystemWed Nov 14 01:16:21 GMT [raid.disk.missing:info]: Disk 3a.10.4 Shelf 10 Bay 4[NETAPP X412_S15K7560A15 NA00] S/N [6SL3RP8A0000N2402TVC] is missing from thesystemWed Nov 14 01:16:21 GMT [raid.disk.missilogin as: p375290p375290@10.21.20.56s password:RLM akfiler12> ?date - print date and timeexit - exit from the RLM command line interfaceevents - print system events and event informationhelp - print command helppriv - show and set user moderlm - commands to control the RLMrsa - commands for Remote Support Agentsystem - commands to control the systemversion - print RLM version
    • RLM akfiler12> system consoleType Ctrl-D to exit.Data ONTAP (akfiler12.ak.providence.org)login: p375290Password:Tue Nov 13 16:32:47 AKST [akfiler12: RLM_console_login:info]: p375290 logged infrom RLMakfiler12> sysconfig -a NetApp Release 7.3.6: Thu Jul 7 01:48:12 PDT 2011 System ID: 0151729197 (akfiler12); partner ID: 0151729165 (akfiler13) System Serial Number: 700000071691 (akfiler12) System Rev: A0 System Storage Configuration: Multi-Path HA System ACP Connectivity: NA slot 0: System Board 2.6 GHz (System Board XV A4) Model Name: FAS3160 Part Number: 110-00119 Revision: A4 Serial Number: 734253 BIOS version: 4.4.0 Loader version: 1.8 Agent FW version: 3 Processors: 4 Processor ID: 0x40f13 Microcode Version: 0x0 Memory Size: 8192 MB Memory Attributes: Node Interleaving Bank Interleaving Hoisting Chipkill ECC CMOS RAM Status: OK Controller: A Remote LAN Module Status: Online Part Number: 110-00057 Revision: G0 Serial Number: 912333 Firmware Version: 4.0 Mgmt MAC Address: 00:A0:98:12:1A:31 Ethernet Link: up, 100Mb, full duplex, auto-neg complete Using DHCP: no IPv4 configuration: IP Address: 10.21.20.56 Netmask: 255.255.255.0 Gateway: 10.21.20.1 IPv6 configuration: Disabled slot 0: Internal BGE 10/100 Ethernet Controller e0M MAC Address: 00:a0:98:12:4c:aa (auto-unknown-cfg_down) Device Type: BCM5703C_B0 memory mapped I/O base 0x00000000c0e00000, size 0x10000 slot 0: BGE 10/100/1000 Ethernet Controller e0a MAC Address: 00:a0:98:12:4c:a8 (auto-1000t-fd-up) e0b MAC Address: 00:a0:98:12:4c:a9 (auto-1000t-fd-up) Device Type: BCM5715C memory mapped I/O base 0x00000000c0400000, size 0x10000 memory mapped I/O base 0x00000000c0410000, size 0x10000 memory mapped I/O base 0x00000000c0420000, size 0x10000 memory mapped I/O base 0x00000000c0430000, size 0x10000 slot 0: NVRAM (NetApp NVRAM VII) Revision: A4 Serial Number: 734253 Memory Size: 2048 MB
    • Battery Status: OK (3924 mV) Charger Status: ON Running Firmware: 4 (4.8.940) LIDs: 0x0004[0x0005], 0x0006[0x0007] Cluster Interconnect Port 1: 4x copper Cluster Interconnect Port 2: 4x copper memory mapped I/O base 0xc4000000, size 0x100000 prefetchable memory base 0xc3800000, size 0x800000 prefetchable memory base 0x3100000000, size 0x80000000 slot 0: FC Host Adapter 0a (QLogic 2432 rev. 2, L-port, <UP>) Firmware rev: 4.5.2 Host Loop Id: 7 FC Node Name: 5:00a:098200:0369df SFP Vendor: FINISAR CORP. SFP Part Number: FTLF8524P2BNV SFP Serial Number: PG10S13 SFP Capabilities: 1, 2 or 4 Gbit Link Data Rate: 4 Gbit 16 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLY0HK7C) 17 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGRJEC) 18 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWDZ2DC) 19 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWEGTAC) 20 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWEL9KC) 21 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWA4T3C) 22 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGLJTC) 23 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGR5RC) 24 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWDVURC) 25 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWELGYC) 26 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGL94C) 27 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGL8JC) 28 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGRBDC) 29 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGRHJC) 32 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0U8J) 33 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJAEMJ) 34 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXGM2NJ) 35 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXDPDNJ) 36 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0AUJ) 37 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXH72DJ) 38 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJYPKJ) 39 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJYJLJ) 40 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXD4E7J)
    • 41 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXHZRKJ) 42 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXHZMLJ) 43 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXBAY3J) 44 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXGM40J) 45 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJRK8J) Shelf 1: ESH4 Firmware rev. ESH A: 14 ESH B: 14 Shelf 2: ESH4 Firmware rev. ESH A: 14 ESH B: 14 I/O base 0x0000000000000000, size 0x0 memory mapped I/O base 0x00000000c0c00000, size 0x4000 slot 0: FC Host Adapter 0b (QLogic 2432 rev. 2, L-port, <UP>) Firmware rev: 4.5.2 Host Loop Id: 7 FC Node Name: 5:00a:098300:0369df SFP Vendor: FINISAR CORP. SFP Part Number: FTLF8524P2BNV SFP Serial Number: PG10ST0 SFP Capabilities: 1, 2 or 4 Gbit Link Data Rate: 4 Gbit 16 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWDZX6C) 17 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWE02VC) 18 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWDZV8C) 19 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWDZWYC) 20 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWE000C) 21 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWEKVPC) 22 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWDZW7C) 23 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWBLMNE) 24 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWBLL2E) 25 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWBLN4E) 26 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWBLMRE) 27 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLW899PC) 28 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLW85H5C) 29 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLW85HSC) 32 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0NRJ) 33 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0PEJ) 34 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0HMJ) 35 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXHAJVJ) 36 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0HYJ) 37 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJZPRJ) 38 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect
    • (JTXJYEHJ) 39 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXG694J) 40 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0DWJ) 41 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXHZLXJ) 42 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0NXJ) 43 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ9L5J) 44 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ107J) 45 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJRVGJ) Shelf 1: ESH4 Firmware rev. ESH A: 14 ESH B: 14 Shelf 2: ESH4 Firmware rev. ESH A: 14 ESH B: 14 I/O base 0x0000000000000000, size 0x0 memory mapped I/O base 0x00000000c0c04000, size 0x4000 slot 0: FC Host Adapter 0c (QLogic 2432 rev. 2, L-port, <UP>) Firmware rev: 4.5.2 Host Loop Id: 7 FC Node Name: 5:00a:098000:0369df SFP Vendor: FINISAR CORP. SFP Part Number: FTLF8524P2BNV SFP Serial Number: PG10RXF SFP Capabilities: 1, 2 or 4 Gbit Link Data Rate: 4 Gbit 16 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWDZX6C) 17 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWE02VC) 18 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWDZV8C) 19 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWDZWYC) 20 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWE000C) 21 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWEKVPC) 22 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWDZW7C) 23 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWBLMNE) 24 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWBLL2E) 25 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWBLN4E) 26 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWBLMRE) 27 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLW899PC) 28 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLW85H5C) 29 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLW85HSC) 32 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0NRJ) 33 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0PEJ) 34 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0HMJ) 35 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXHAJVJ)
    • 36 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0HYJ) 37 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJZPRJ) 38 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJYEHJ) 39 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXG694J) 40 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0DWJ) 41 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXHZLXJ) 42 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0NXJ) 43 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ9L5J) 44 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ107J) 45 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJRVGJ) Shelf 1: ESH4 Firmware rev. ESH A: 14 ESH B: 14 Shelf 2: ESH4 Firmware rev. ESH A: 14 ESH B: 14 I/O base 0x0000000000000000, size 0x0 memory mapped I/O base 0x00000000c0a00000, size 0x4000 slot 0: FC Host Adapter 0d (QLogic 2432 rev. 2, L-port, <UP>) Firmware rev: 4.5.2 Host Loop Id: 7 FC Node Name: 5:00a:098100:0369df SFP Vendor: FINISAR CORP. SFP Part Number: FTLF8524P2BNV SFP Serial Number: PG10RYR SFP Capabilities: 1, 2 or 4 Gbit Link Data Rate: 4 Gbit 16 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLY0HK7C) 17 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGRJEC) 18 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWDZ2DC) 19 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWEGTAC) 20 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWEL9KC) 21 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWA4T3C) 22 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGLJTC) 23 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGR5RC) 24 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWDVURC) 25 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWELGYC) 26 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGL94C) 27 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGL8JC) 28 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGRBDC) 29 : NETAPP X279_HVPBP288F15 NA02 272.0GB 520B/sect(JLWGRHJC) 32 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0U8J) 33 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect
    • (JTXJAEMJ) 34 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXGM2NJ) 35 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXDPDNJ) 36 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJ0AUJ) 37 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXH72DJ) 38 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJYPKJ) 39 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJYJLJ) 40 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXD4E7J) 41 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXHZRKJ) 42 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXHZMLJ) 43 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXBAY3J) 44 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXGM40J) 45 : NETAPP X279_HVIPC288F15 NA02 272.0GB 520B/sect(JTXJRK8J) Shelf 1: ESH4 Firmware rev. ESH A: 14 ESH B: 14 Shelf 2: ESH4 Firmware rev. ESH A: 14 ESH B: 14 I/O base 0x0000000000000000, size 0x0 memory mapped I/O base 0x00000000c0a04000, size 0x4000 slot 0: ATA/IDE Adapter 0e (0x00000000000001f0) 0e.0 STEC NACF1GM1U-B11 01/17/07 977MB512B/sect (STI1MML409177165039) I/O base 0x0000000000001018, size 0x8 I/O base 0x0000000000001024, size 0x4 I/O base 0x0000000000001010, size 0x8 I/O base 0x0000000000001020, size 0x4 I/O base 0x0000000000001000, size 0x10 slot 1: Performance Acceleration Module I State: Enabled Memory Size: 16 GB Model Name: X1936A-R5 Serial Number: 202848 Part Number: 111-00360 Board Revision: B1 FPGA Release: 1.0 FPGA Build: 200706131558 DIMM Size Part Number Serial Number Status 1: 4 GB NHM517R22503F-D32K 16-00-0922-75A60068 Enabled 2: 4 GB NHM517R22503F-D32K 16-00-0922-75B60072 Enabled 3: 4 GB NHM517R22503F-D32K 16-00-0922-75C60018 Enabled 4: 4 GB NHM517R22503F-D32K 16-00-0922-75E60022 Enabled memory mapped I/O base 0x00000000c3000000, size 0x10000 memory mapped I/O base 0x00000000c1000000, size0x1000000 memory mapped I/O base 0x00000000c2000000, size0x1000000 slot 3: SAS Host Adapter 3a (PMC-Sierra PM8001 rev. C, SAS, <UP>) MFG Part Number: USI 6086-000010-04 rev. MP V4.6 Part number: 111-00341+D0 Serial number: ST074C2H0FC Date Code: 20120217 Firmware rev: 01.11.00.00 Base WWN: 5:00e004:0001d2c:e8 Phy State: [0] Enabled, 3.0 Gb/s
    • [1] Enabled, 3.0 Gb/s [2] Enabled, 3.0 Gb/s [3] Enabled, 3.0 Gb/s QSFP Vendor: Molex Inc. QSFP Part Number: 112-00178+A0 QSFP Type: Passive Copper 5m ID:00 QSFP Serial Number: 211620191 10.0 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SYV30000N2406M4C) 10.1 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S1HQ0000N239HAWN) 10.2 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3NLK80000N239HAGX) 10.3 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3PLKS0000N238HXQ8) 10.4 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RP8A0000N2402TVC) 10.5 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S4CS0000N239FMKB) 10.6 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RVBS0000N23992NC) 10.7 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SL2H0000N240CYH3) 10.8 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SJZ70000N2406NG0) 10.9 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RV2N0000N2394T49) 10.10: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL2Y3L40000N2257M0S) 10.11: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SKQ70000N240009W) 10.12: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SSMR0000N2407YCM) 10.13: NETAPP X412_HVIPC560A15 NA01 560.0GB 520B/sect(J9YESPBL) 10.14: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S8ED0000N239H9SB) 10.15: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3T6HM0000N2401AY2) 10.16: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S9R90000N239DEYZ) 10.17: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S0C70000N238KV7Y) 10.18: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3F3F30000N240027Z) 10.19: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RF0K0000N239H59Y) 10.20: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RZT60000N24003YQ) 10.21: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RTS20000N239CP1A) 10.22: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3PFL70000N2395CF0) 10.23: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S5C60000N2401FT9) 11.0 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RLSC0000N2401CX4) 11.1 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RV360000N2394T2J) 11.2 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3R1BS0000N239GVNT) 11.3 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RSAR0000N2398EHS) 11.4 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect
    • (6SL3RM7M0000N2401E19) 11.5 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S65B0000N2394U6N) 11.6 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3Q4RE0000N239FL0Q) 11.7 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3PKCJ0000N239GSCD) 11.8 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3MHSN0000N238HX6U) 11.9 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3PEH90000N2402WT4) 11.10: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3KABA0000N239CX2H) 11.11: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RTMK0000N238H4HW) 11.12: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RHBT0000N2398H8A) 11.13: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S6NF0000N240060G) 11.14: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RZ6K0000N23906N9) 11.15: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SB830000N2394XV7) 11.16: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3QCG00000N239GU7U) 11.17: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SJGJ0000N2401EN2) 11.18: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S5BY0000N2401F2H) 11.19: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SD270000N2398G0U) 11.20: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S95X0000N238HV92) 11.21: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S7M70000N239FJU4) 11.22: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RSQM0000N2398BDY) 11.23: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RTYF0000N239GTZL) Shelf 10: IOM3 Firmware rev. IOM3 A: 0132 IOM3 B: 0132 Shelf 11: IOM3 Firmware rev. IOM3 A: 0132 IOM3 B: 0132 slot 3: SAS Host Adapter 3b (PMC-Sierra PM8001 rev. C, SAS, <OFFLINE(hard)>) MFG Part Number: USI 6086-000010-04 rev. MP V4.6 Part number: 111-00341+D0 Serial number: ST074C2H0FC Date Code: 20120217 Firmware rev: 01.11.00.00 Base WWN: 5:00e004:0001d2c:ec Phy State: [4] Disabled [5] Disabled [6] Disabled [7] Disabled QSFP Vendor: not available QSFP Part Number: not available QSFP Type: not available QSFP Serial Number: not available memory mapped I/O base 0x00000000c0200000, size 0x10000 memory mapped I/O base 0x00000000c0210000, size 0x10000 memory mapped I/O base 0x00000000c0220000, size 0x10000 memory mapped I/O base 0x00000000c0230000, size 0x10000 slot 3: SAS Host Adapter 3c (PMC-Sierra PM8001 rev. C, SAS, <OFFLINE(hard)>) MFG Part Number: USI 6086-000010-04 rev. MP V4.6
    • Part number: 111-00341+D0 Serial number: ST074C2H0FC Date Code: 20120217 Firmware rev: 01.11.00.00 Base WWN: 5:00e004:0001d2c:e0 Phy State: [0] Disabled [1] Disabled [2] Disabled [3] Disabled QSFP Vendor: not available QSFP Part Number: not available QSFP Type: not available QSFP Serial Number: not available slot 3: SAS Host Adapter 3d (PMC-Sierra PM8001 rev. C, SAS, <UP>) MFG Part Number: USI 6086-000010-04 rev. MP V4.6 Part number: 111-00341+D0 Serial number: ST074C2H0FC Date Code: 20120217 Firmware rev: 01.11.00.00 Base WWN: 5:00e004:0001d2c:e4 Phy State: [4] Enabled, 3.0 Gb/s [5] Enabled, 3.0 Gb/s [6] Enabled, 3.0 Gb/s [7] Enabled, 3.0 Gb/s QSFP Vendor: Molex Inc. QSFP Part Number: 112-00178+A0 QSFP Type: Passive Copper 5m ID:00 QSFP Serial Number: 211620177 10.0 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SYV30000N2406M4C) 10.1 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S1HQ0000N239HAWN) 10.2 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3NLK80000N239HAGX) 10.3 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3PLKS0000N238HXQ8) 10.4 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RP8A0000N2402TVC) 10.5 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S4CS0000N239FMKB) 10.6 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RVBS0000N23992NC) 10.7 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SL2H0000N240CYH3) 10.8 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SJZ70000N2406NG0) 10.9 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RV2N0000N2394T49) 10.10: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL2Y3L40000N2257M0S) 10.11: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SKQ70000N240009W) 10.12: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SSMR0000N2407YCM) 10.13: NETAPP X412_HVIPC560A15 NA01 560.0GB 520B/sect(J9YESPBL) 10.14: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S8ED0000N239H9SB) 10.15: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3T6HM0000N2401AY2) 10.16: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S9R90000N239DEYZ) 10.17: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S0C70000N238KV7Y)
    • 10.18: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3F3F30000N240027Z) 10.19: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RF0K0000N239H59Y) 10.20: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RZT60000N24003YQ) 10.21: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RTS20000N239CP1A) 10.22: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3PFL70000N2395CF0) 10.23: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S5C60000N2401FT9) 11.0 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RLSC0000N2401CX4) 11.1 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RV360000N2394T2J) 11.2 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3R1BS0000N239GVNT) 11.3 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RSAR0000N2398EHS) 11.4 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RM7M0000N2401E19) 11.5 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S65B0000N2394U6N) 11.6 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3Q4RE0000N239FL0Q) 11.7 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3PKCJ0000N239GSCD) 11.8 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3MHSN0000N238HX6U) 11.9 : NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3PEH90000N2402WT4) 11.10: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3KABA0000N239CX2H) 11.11: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RTMK0000N238H4HW) 11.12: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RHBT0000N2398H8A) 11.13: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S6NF0000N240060G) 11.14: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RZ6K0000N23906N9) 11.15: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SB830000N2394XV7) 11.16: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3QCG00000N239GU7U) 11.17: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SJGJ0000N2401EN2) 11.18: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S5BY0000N2401F2H) 11.19: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3SD270000N2398G0U) 11.20: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S95X0000N238HV92) 11.21: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3S7M70000N239FJU4) 11.22: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RSQM0000N2398BDY) 11.23: NETAPP X412_S15K7560A15 NA00 560.0GB 520B/sect(6SL3RTYF0000N239GTZL) Shelf 10: IOM3 Firmware rev. IOM3 A: 0132 IOM3 B: 0132 Shelf 11: IOM3 Firmware rev. IOM3 A: 0132 IOM3 B: 0132 memory mapped I/O base 0x00000000c0000000, size 0x10000 memory mapped I/O base 0x00000000c0010000, size 0x10000
    • memory mapped I/O base 0x00000000c0020000, size 0x10000 memory mapped I/O base 0x00000000c0030000, size 0x10000 slot 4: Quad 10/100/1000 Ethernet Controller G20 e4a MAC Address: 00:15:17:c0:25:94 (auto-1000t-fd-up) e4b MAC Address: 00:15:17:c0:25:95 (auto-1000t-fd-up) e4c MAC Address: 00:15:17:c0:25:96 (auto-unknown-cfg_down) e4d MAC Address: 00:15:17:c0:25:97 (auto-unknown-cfg_down) Device Type: Rev 6, PBA D45774-008 memory mapped I/O base 0x00000000c0600000, size 0x20000 memory mapped I/O base 0x00000000c0620000, size 0x20000 I/O base 0x0000000000000000, size 0x0 memory mapped I/O base 0x00000000c0640000, size 0x20000 memory mapped I/O base 0x00000000c0660000, size 0x20000 I/O base 0x0000000000000000, size 0x0 memory mapped I/O base 0x00000000c0800000, size 0x20000 memory mapped I/O base 0x00000000c0820000, size 0x20000 I/O base 0x0000000000000000, size 0x0 memory mapped I/O base 0x00000000c0840000, size 0x20000 memory mapped I/O base 0x00000000c0860000, size 0x20000 I/O base 0x0000000000000000, size 0x0akfiler12> vol status Volume State Status Options vol0 online raid_dp, flex root vmpg online raid_dp, flex guarantee=none sis pocvm1 online raid_dp, flex guarantee=none sis Dragon online raid_dp, flex create_ucode=on, sis convert_ucode=on,guarantee=none, fractional_reserve=0akfiler12onbase online raid_dp, flex create_ucode=on, sis convert_ucode=on,guarantee=none, fractional_reserve=0 ISO online raid_dp, flex guarantee=none sis vm04 online raid_dp, flex guarantee=none sis vm06 online raid_dp, flex create_ucode=on, sis convert_ucode=on,guarantee=none, fractional_reserve=0 vm01 online raid_dp, flex guarantee=none sis vm02 online raid_dp, flex guarantee=none sis vm03 online raid_dp, flex guarantee=none, sis fractional_reserve=0akfiler12> df -AAggregate kbytes used avail capacityaggr1 10837619288 1265112748 9572506540 12%aggr1/.snapshot 0 0 0 ---%aggr0 5514438760 4746399540 768039220 86%aggr0/.snapshot 0 0 0 ---%akfiler12> aggr status Aggr State Status Options aggr0 online raid_dp, aggr root aggr1 online raid_dp, aggr nosnap=on, raidsize=23akfiler12> rdfile /etc/messagesSun Nov 11 00:00:00 AKST [akfiler12: kern.log.rotate:notice]: System akfiler12(ID 0151729197) is running NetApp Release 7.3.6Sun Nov 11 00:00:03 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.
    • Sun Nov 11 00:00:46 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of nightly snapshot (previous auto-delete snap create still inprogress)Sun Nov 11 01:00:00 AKST [akfiler12: kern.uptime.filer:info]: 1:00am up 199days, 5:44 17943788650 NFS ops, 8487405 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 01:00:10 AKST [akfiler12: raid.rg.scrub.resume:notice]:/aggr0/plex0/rg0: resuming scrub at stripe 51693808 (74% complete)Sun Nov 11 01:00:14 AKST [akfiler12: raid.rg.scrub.resume:notice]:/aggr0/plex0/rg1: resuming scrub at stripe 45466536 (65% complete)Sun Nov 11 01:00:14 AKST [akfiler12: raid.rg.scrub.start:notice]:/aggr1/plex0/rg0: starting scrubSun Nov 11 01:00:16 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 01:27:14 AKST [akfiler12: asup.smtp.sent:notice]: ClusterNotification mail sent: Cluster Notification from akfiler12 (WEEKLY_LOG) INFOSun Nov 11 01:27:34 AKST [akfiler12: asup.smtp.sent:notice]: ClusterNotification mail sent: Cluster Notification from akfiler12 (PERFORMANCE DATA)INFOSun Nov 11 02:00:00 AKST [akfiler12: kern.uptime.filer:info]: 2:00am up 199days, 6:44 17956114773 NFS ops, 8487635 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 02:00:03 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 03:00:00 AKST [akfiler12: kern.uptime.filer:info]: 3:00am up 199days, 7:44 17967100191 NFS ops, 8487891 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 03:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 04:00:00 AKST [akfiler12: kern.uptime.filer:info]: 4:00am up 199days, 8:44 17975420248 NFS ops, 8488173 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 04:00:04 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 05:00:00 AKST [akfiler12: kern.uptime.filer:info]: 5:00am up 199days, 9:44 17979509075 NFS ops, 8488485 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 05:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 05:00:50 AKST [akfiler12: app.log.info:info]: wn6147: Monitoring andHost Configuration sub-plugin 4.0: (0) discovery: Storage discovery task foundthis controller to be connected to an ESX host managed by vCenter or manuallyentered.Sun Nov 11 05:01:29 AKST [akfiler12: app.log.info:info]: wn6147: Monitoring andHost Configuration sub-plugin 4.0: (0) VmDiscovery:wn6147.ak.providence.org(404):winNetStandard64Guest=2,other26xLinux64Guest=1,winLonghorn64Guest=3,otherLinuxGuest=4,winNetEnterpriseGuest=8,winLonghornGuest=55,rhel4Guest=1,otherGuest=1,winXPProGuest=62,win2000ServGuest=9,rhel3Guest=1,slesGuest=1Sun Nov 11 05:01:31 AKST [akfiler12: app.log.info:info]: wn6147: Monitoring andHost Configuration sub-plugin 4.0: (0) VmDiscovery:wn6147.ak.providence.org(404):windows7_64Guest=1,winNetStandardGuest=197,rhel5Guest=5,windows7Server64Guest=53Sun Nov 11 05:07:27 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #54591842 during scrubSun Nov 11 05:09:01 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #54677029 during scrubSun Nov 11 05:09:12 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPP
    • X279_HVPBP288F15 NA02] S/N [JLWDVURC], block #54686162 during scrubSun Nov 11 05:19:23 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #55379800 during scrubSun Nov 11 05:20:59 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #55528198 during scrubSun Nov 11 05:23:07 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #55761034 during scrubSun Nov 11 05:26:16 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #55892168 during scrubSun Nov 11 05:27:18 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #55892176 during scrubSun Nov 11 05:29:09 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #55987242 during scrubSun Nov 11 05:29:20 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #55892178 during scrubSun Nov 11 05:31:12 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56172831 during scrubSun Nov 11 05:31:18 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56182573 during scrubSun Nov 11 05:31:58 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56227770 during scrubSun Nov 11 05:32:36 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56276216 during scrubSun Nov 11 05:32:37 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56276632 during scrubSun Nov 11 05:33:16 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56313925 during scrubSun Nov 11 05:34:25 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56407038 during scrubSun Nov 11 05:35:25 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56503047 during scrubSun Nov 11 05:36:48 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPP
    • X279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56651449 during scrubSun Nov 11 05:38:45 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56875945 during scrubSun Nov 11 05:38:48 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56878017 during scrubSun Nov 11 05:39:28 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56931145 during scrubSun Nov 11 05:40:23 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #57017356 during scrubSun Nov 11 05:40:23 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #57017357 during scrubSun Nov 11 05:42:10 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #57213853 during scrubSun Nov 11 05:44:30 AKST [akfiler12:raid.tetris.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56502010 during stripe writeSun Nov 11 05:45:28 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #57633530 during scrubSun Nov 11 05:47:00 AKST [akfiler12:raid.rg.scrub.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #57905294 during scrubSun Nov 11 05:55:17 AKST [akfiler12: raid.rg.scrub.done:notice]:/aggr1/plex0/rg0: scrub completed in 4:55:03.33Sun Nov 11 05:55:17 AKST [akfiler12: raid.rg.scrub.summary.pi:notice]: Scrubfound 0 parity inconsistencies in /aggr1/plex0/rg0.Sun Nov 11 05:55:17 AKST [akfiler12: raid.rg.scrub.summary.cksum:notice]: Scrubfound 0 checksum errors in /aggr1/plex0/rg0.Sun Nov 11 05:55:17 AKST [akfiler12: raid.rg.scrub.summary.media:notice]: Scrubfound 0 media errors in /aggr1/plex0/rg0.Sun Nov 11 05:55:57 AKST [akfiler12: raid.rg.scrub.done:notice]:/aggr0/plex0/rg1: scrub completed in 4:55:44.93Sun Nov 11 05:55:57 AKST [akfiler12: raid.rg.scrub.summary.pi:notice]: Scrubfound 0 parity inconsistencies in /aggr0/plex0/rg1.Sun Nov 11 05:55:57 AKST [akfiler12: raid.rg.scrub.summary.cksum:notice]: Scrubfound 0 checksum errors in /aggr0/plex0/rg1.Sun Nov 11 05:55:57 AKST [akfiler12: raid.rg.scrub.summary.media:notice]: Scrubfound 0 media errors in /aggr0/plex0/rg1.Sun Nov 11 06:00:00 AKST [akfiler12: kern.uptime.filer:info]: 6:00am up 199days, 10:44 17982160612 NFS ops, 8488805 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 06:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 06:36:09 AKST [akfiler12: raid.rg.scrub.done:notice]:/aggr0/plex0/rg0: scrub completed in 5:36:01.10Sun Nov 11 06:36:09 AKST [akfiler12: raid.rg.scrub.summary.pi:notice]: Scrubfound 0 parity inconsistencies in /aggr0/plex0/rg0.Sun Nov 11 06:36:09 AKST [akfiler12: raid.rg.scrub.summary.cksum:notice]: Scrubfound 0 checksum errors in /aggr0/plex0/rg0.
    • Sun Nov 11 06:36:09 AKST [akfiler12: raid.rg.scrub.summary.media:warning]: Scrubfound 34 media errors in /aggr0/plex0/rg0.Sun Nov 11 06:36:28 AKST [akfiler12: asup.smtp.sent:notice]: ClusterNotification mail sent: Cluster Notification from akfiler12 (DISK SCRUB SUMMARY- no customer action required) INFOSun Nov 11 07:00:00 AKST [akfiler12: kern.uptime.filer:info]: 7:00am up 199days, 11:44 17983725512 NFS ops, 8489103 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 07:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 08:00:00 AKST [akfiler12: kern.uptime.filer:info]: 8:00am up 199days, 12:44 17985355696 NFS ops, 8489345 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 08:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 09:00:00 AKST [akfiler12: kern.uptime.filer:info]: 9:00am up 199days, 13:44 17987128934 NFS ops, 8489645 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 09:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 09:00:45 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of hourly snapshot (previous auto-delete snap create still inprogress)Sun Nov 11 10:00:00 AKST [akfiler12: kern.uptime.filer:info]: 10:00am up 199days, 14:44 17988828418 NFS ops, 8489947 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 10:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 11:00:00 AKST [akfiler12: kern.uptime.filer:info]: 11:00am up 199days, 15:44 17990493359 NFS ops, 8490259 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 11:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 12:00:00 AKST [akfiler12: kern.uptime.filer:info]: 12:00pm up 199days, 16:44 17992105875 NFS ops, 8490536 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 12:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 12:00:25 AKST [akfiler12:raid.read.media.recommend.reassign.err:info]: Block recommended for reassignmenton Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPP X279_HVPBP288F15 NA02]S/N [JLWDVURC], block #56838135Sun Nov 11 12:00:25 AKST [akfiler12: raid.read.media.err:info]: Read error onDisk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPP X279_HVPBP288F15 NA02] S/N[JLWDVURC], block #56838137Sun Nov 11 12:00:27 AKST [akfiler12:raid.read.media.recommend.reassign.err:info]: Block recommended for reassignmenton Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPP X279_HVPBP288F15 NA02]S/N [JLWDVURC], block #56838134Sun Nov 11 13:00:00 AKST [akfiler12: kern.uptime.filer:info]: 1:00pm up 199days, 17:44 17993828110 NFS ops, 8490836 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 13:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 14:00:00 AKST [akfiler12: kern.uptime.filer:info]: 2:00pm up 199days, 18:44 17995471081 NFS ops, 8491119 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 14:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 14:00:45 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of hourly snapshot (previous auto-delete snap create still inprogress)Sun Nov 11 15:00:00 AKST [akfiler12: kern.uptime.filer:info]: 3:00pm up 199days, 19:44 17997092755 NFS ops, 8491422 CIFS ops, 145 HTTP ops, 0 FCP ops, 0
    • iSCSI opsSun Nov 11 15:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 16:00:00 AKST [akfiler12: kern.uptime.filer:info]: 4:00pm up 199days, 20:44 17999836810 NFS ops, 8491746 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 16:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 17:00:00 AKST [akfiler12: kern.uptime.filer:info]: 5:00pm up 199days, 21:44 18001973444 NFS ops, 8491992 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 17:00:01 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 18:00:00 AKST [akfiler12: kern.uptime.filer:info]: 6:00pm up 199days, 22:44 18003727041 NFS ops, 8492289 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 18:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 19:00:00 AKST [akfiler12: kern.uptime.filer:info]: 7:00pm up 199days, 23:44 18008923309 NFS ops, 8492541 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 19:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 19:00:30 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of hourly snapshot (previous auto-delete snap create still inprogress)Sun Nov 11 19:15:51 AKST [akfiler12: iomem.ecc.summary:warning]: Caching modulein slot 1 has detected a total of 12316 correctable and 9893 uncorrectablememory errors.Sun Nov 11 20:00:00 AKST [akfiler12: kern.uptime.filer:info]: 8:00pm up 200days, 44 mins, 18012656879 NFS ops, 8492841 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 20:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 20:15:51 AKST [akfiler12: iomem.ecc.summary:warning]: Caching modulein slot 1 has detected a total of 12422 correctable and 10110 uncorrectablememory errors.Sun Nov 11 21:00:00 AKST [akfiler12: kern.uptime.filer:info]: 9:00pm up 200days, 1:44 18015599375 NFS ops, 8493142 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 21:00:03 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 22:00:00 AKST [akfiler12: kern.uptime.filer:info]: 10:00pm up 200days, 2:44 18017654168 NFS ops, 8493424 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 22:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Sun Nov 11 23:00:00 AKST [akfiler12: kern.uptime.filer:info]: 11:00pm up 200days, 3:44 18019546493 NFS ops, 8493746 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsSun Nov 11 23:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 00:00:00 AKST [akfiler12: kern.uptime.filer:info]: 12:00am up 200days, 4:44 18021300457 NFS ops, 8493998 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 00:00:03 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 00:00:30 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of nightly snapshot (previous auto-delete snap create still inprogress)Mon Nov 12 01:00:00 AKST [akfiler12: kern.uptime.filer:info]: 1:00am up 200days, 5:44 18027766315 NFS ops, 8494296 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 01:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScale
    • WAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 02:00:00 AKST [akfiler12: kern.uptime.filer:info]: 2:00am up 200days, 6:44 18035872266 NFS ops, 8494549 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 02:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 03:00:00 AKST [akfiler12: kern.uptime.filer:info]: 3:00am up 200days, 7:44 18038093428 NFS ops, 8494845 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 03:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 04:00:00 AKST [akfiler12: kern.uptime.filer:info]: 4:00am up 200days, 8:44 18039850389 NFS ops, 8495131 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 04:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 05:00:00 AKST [akfiler12: kern.uptime.filer:info]: 5:00am up 200days, 9:44 18041539750 NFS ops, 8495438 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 05:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 05:00:50 AKST [akfiler12: app.log.info:info]: wn6147: Monitoring andHost Configuration sub-plugin 4.0: (0) discovery: Storage discovery task foundthis controller to be connected to an ESX host managed by vCenter or manuallyentered.Mon Nov 12 05:01:28 AKST [akfiler12: app.log.info:info]: wn6147: Monitoring andHost Configuration sub-plugin 4.0: (0) VmDiscovery:wn6147.ak.providence.org(404):winNetStandard64Guest=2,other26xLinux64Guest=1,winLonghorn64Guest=3,otherLinuxGuest=4,winNetEnterpriseGuest=8,winLonghornGuest=55,rhel4Guest=1,otherGuest=1,winXPProGuest=62,win2000ServGuest=9,rhel3Guest=1,slesGuest=1Mon Nov 12 05:01:30 AKST [akfiler12: app.log.info:info]: wn6147: Monitoring andHost Configuration sub-plugin 4.0: (0) VmDiscovery:wn6147.ak.providence.org(404):windows7_64Guest=1,winNetStandardGuest=197,rhel5Guest=5,windows7Server64Guest=53Mon Nov 12 06:00:00 AKST [akfiler12: kern.uptime.filer:info]: 6:00am up 200days, 10:44 18043428598 NFS ops, 8495722 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 06:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 07:00:00 AKST [akfiler12: kern.uptime.filer:info]: 7:00am up 200days, 11:44 18045106374 NFS ops, 8496037 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 07:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 08:00:00 AKST [akfiler12: kern.uptime.filer:info]: 8:00am up 200days, 12:44 18047722860 NFS ops, 8496350 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 08:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 09:00:00 AKST [akfiler12: kern.uptime.filer:info]: 9:00am up 200days, 13:44 18050428873 NFS ops, 8496572 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 09:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 09:00:31 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of hourly snapshot (previous auto-delete snap create still inprogress)Mon Nov 12 10:00:00 AKST [akfiler12: kern.uptime.filer:info]: 10:00am up 200days, 14:44 18053210427 NFS ops, 8496874 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 10:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 11:00:00 AKST [akfiler12: kern.uptime.filer:info]: 11:00am up 200
    • days, 15:44 18056910463 NFS ops, 8497177 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 11:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 12:00:00 AKST [akfiler12: kern.uptime.filer:info]: 12:00pm up 200days, 16:44 18061308270 NFS ops, 8497536 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 12:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 13:00:00 AKST [akfiler12: kern.uptime.filer:info]: 1:00pm up 200days, 17:44 18065548814 NFS ops, 8497837 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 13:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 13:30:02 AKST [akfiler12: RLM_console_login:info]: p375290 logged infrom RLMMon Nov 12 14:00:00 AKST [akfiler12: kern.uptime.filer:info]: 2:00pm up 200days, 18:44 18069923123 NFS ops, 8498119 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 14:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 14:00:30 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of hourly snapshot (previous auto-delete snap create still inprogress)Mon Nov 12 15:00:00 AKST [akfiler12: kern.uptime.filer:info]: 3:00pm up 200days, 19:44 18072321633 NFS ops, 8498414 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 15:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 15:31:17 AKST [akfiler12:raid.tetris.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #57306564 during stripe writeMon Nov 12 16:00:00 AKST [akfiler12: kern.uptime.filer:info]: 4:00pm up 200days, 20:44 18074483058 NFS ops, 8498703 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 16:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 16:46:16 AKST [akfiler12:raid.tetris.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #56876043 during stripe writeMon Nov 12 17:00:00 AKST [akfiler12: kern.uptime.filer:info]: 5:00pm up 200days, 21:44 18077691196 NFS ops, 8498956 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 17:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 18:00:00 AKST [akfiler12: kern.uptime.filer:info]: 6:00pm up 200days, 22:44 18079787579 NFS ops, 8499266 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 18:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 19:00:00 AKST [akfiler12: kern.uptime.filer:info]: 7:00pm up 200days, 23:44 18083324551 NFS ops, 8499566 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 19:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 19:00:30 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of hourly snapshot (previous auto-delete snap create still inprogress)Mon Nov 12 20:00:00 AKST [akfiler12: kern.uptime.filer:info]: 8:00pm up 201days, 44 mins, 18087315819 NFS ops, 8499860 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 20:00:03 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScale
    • WAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 21:00:00 AKST [akfiler12: kern.uptime.filer:info]: 9:00pm up 201days, 1:44 18090910495 NFS ops, 8500160 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 21:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 22:00:00 AKST [akfiler12: kern.uptime.filer:info]: 10:00pm up 201days, 2:44 18092694494 NFS ops, 8500436 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 22:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Mon Nov 12 23:00:00 AKST [akfiler12: kern.uptime.filer:info]: 11:00pm up 201days, 3:44 18094668356 NFS ops, 8500693 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsMon Nov 12 23:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 00:00:00 AKST [akfiler12: kern.uptime.filer:info]: 12:00am up 201days, 4:44 18096530067 NFS ops, 8500991 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 00:00:03 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 00:00:30 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of nightly snapshot (previous auto-delete snap create still inprogress)Tue Nov 13 01:00:00 AKST [akfiler12: kern.uptime.filer:info]: 1:00am up 201days, 5:44 18103873223 NFS ops, 8501298 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 01:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 02:00:00 AKST [akfiler12: kern.uptime.filer:info]: 2:00am up 201days, 6:44 18112836572 NFS ops, 8501609 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 02:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 02:59:08 AKST [akfiler12:raid.read.media.recommend.reassign.err:info]: Block recommended for reassignmenton Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPP X279_HVPBP288F15 NA02]S/N [JLWDVURC], block #55996677Tue Nov 13 03:00:00 AKST [akfiler12: kern.uptime.filer:info]: 3:00am up 201days, 7:44 18117246236 NFS ops, 8501866 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 03:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 04:00:00 AKST [akfiler12: kern.uptime.filer:info]: 4:00am up 201days, 8:44 18119014127 NFS ops, 8502162 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 04:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 04:15:27 AKST [akfiler12:raid.tetris.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #55155074 during stripe writeTue Nov 13 05:00:00 AKST [akfiler12: kern.uptime.filer:info]: 5:00am up 201days, 9:44 18121807020 NFS ops, 8502468 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 05:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 05:00:51 AKST [akfiler12: app.log.info:info]: wn6147: Monitoring andHost Configuration sub-plugin 4.0: (0) discovery: Storage discovery task foundthis controller to be connected to an ESX host managed by vCenter or manuallyentered.Tue Nov 13 05:01:28 AKST [akfiler12: app.log.info:info]: wn6147: Monitoring andHost Configuration sub-plugin 4.0: (0) VmDiscovery:wn6147.ak.providence.org(405):winNetStandard64Guest=2,other26xLinux64Guest=1,win
    • Longhorn64Guest=3,otherLinuxGuest=4,winNetEnterpriseGuest=8,winLonghornGuest=55,rhel4Guest=1,otherGuest=1,winXPProGuest=62,win2000ServGuest=9,rhel3Guest=1,slesGuest=1Tue Nov 13 05:01:31 AKST [akfiler12: app.log.info:info]: wn6147: Monitoring andHost Configuration sub-plugin 4.0: (0) VmDiscovery:wn6147.ak.providence.org(405):windows7_64Guest=1,winNetStandardGuest=197,rhel5Guest=5,windows7Server64Guest=54Tue Nov 13 05:19:54 AKST [akfiler12:raid.tetris.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #57110162 during stripe writeTue Nov 13 06:00:00 AKST [akfiler12: kern.uptime.filer:info]: 6:00am up 201days, 10:44 18124084052 NFS ops, 8502721 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 06:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 06:31:21 AKST [akfiler12:raid.tetris.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #57855178 during stripe writeTue Nov 13 07:00:00 AKST [akfiler12: kern.uptime.filer:info]: 7:00am up 201days, 11:44 18125795406 NFS ops, 8503008 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 07:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 08:00:00 AKST [akfiler12: kern.uptime.filer:info]: 8:00am up 201days, 12:44 18127653017 NFS ops, 8503311 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 08:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 08:15:40 AKST [akfiler12: iomem.ecc.summary:warning]: Caching modulein slot 1 has detected a total of 13139 correctable and 11645 uncorrectablememory errors.Tue Nov 13 09:00:00 AKST [akfiler12: kern.uptime.filer:info]: 9:00am up 201days, 13:44 18129961988 NFS ops, 8503624 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 09:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 09:00:30 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of hourly snapshot (previous auto-delete snap create still inprogress)Tue Nov 13 10:00:00 AKST [akfiler12: kern.uptime.filer:info]: 10:00am up 201days, 14:44 18132257488 NFS ops, 8503882 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 10:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 11:00:00 AKST [akfiler12: kern.uptime.filer:info]: 11:00am up 201days, 15:44 18135276459 NFS ops, 8504196 CIFS ops, 145 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 11:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 12:00:00 AKST [akfiler12: kern.uptime.filer:info]: 12:00pm up 201days, 16:44 18137806055 NFS ops, 8504527 CIFS ops, 157 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 12:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 13:00:00 AKST [akfiler12: kern.uptime.filer:info]: 1:00pm up 201days, 17:44 18141265505 NFS ops, 8504832 CIFS ops, 157 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 13:00:03 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 14:00:00 AKST [akfiler12: kern.uptime.filer:info]: 2:00pm up 201days, 18:44 18147070479 NFS ops, 8505089 CIFS ops, 157 HTTP ops, 0 FCP ops, 0iSCSI ops
    • Tue Nov 13 14:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 14:00:31 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of hourly snapshot (previous auto-delete snap create still inprogress)Tue Nov 13 15:00:00 AKST [akfiler12: kern.uptime.filer:info]: 3:00pm up 201days, 19:44 18149193778 NFS ops, 8505372 CIFS ops, 157 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 15:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 15:30:21 AKST [akfiler12: RLM_console_login:info]: p375290 logged infrom RLMTue Nov 13 16:00:00 AKST [akfiler12: kern.uptime.filer:info]: 4:00pm up 201days, 20:44 18151427295 NFS ops, 8505746 CIFS ops, 157 HTTP ops, 0 FCP ops, 0iSCSI opsTue Nov 13 16:00:02 AKST [akfiler12: monitor.extCache.failed:warning]: FlexScaleWAFL external cache is unavailable due to an earlier fatal error.Tue Nov 13 16:05:43 AKST [akfiler12: console_login_mgr:info]: netapp logged infrom consoleTue Nov 13 16:11:11 AKST [akfiler12:cf.fsm.nfo.startingGracefulShutdown:warning]: Negotiated failover: startinggraceful shutdown.Tue Nov 13 16:11:17 AKST [akfiler12: kern.shutdown:notice]: System shut downbecause : "reboot".Tue Nov 13 16:11:56 AKST [akfiler12/akfiler13: kern.syslogd.restarted:info]:syslogd: Restarted.Tue Nov 13 16:12:07 AKST [akfiler12/akfiler13:nbt.nbns.registrationComplete:info]: NBT: All CIFS name registrations havecompleted for the partner server.Tue Nov 13 16:12:32 AKST [akfiler12/akfiler13: asup.smtp.sent:notice]: ClusterNotification mail sent: Cluster Notification from akfiler12 (REBOOT (CLUSTERTAKEOVER)) WARNINGWed Nov 14 01:31:03 GMT [localhost: kern.syslogd.error:error]: syslog daemonerror logerror syslogd: Couldnt find address for host name"lx9239.ak.providence.org": Unknown resolver error .Wed Nov 14 01:31:03 GMT [localhost: kern.syslogd.restarted:info]: syslogd:Restarted.Wed Nov 14 01:31:03 GMT [localhost: cf.fsm.takeoverOfPartnerDisabled:notice]:Cluster monitor: takeover of akfiler13 disabled (interconnect error)Wed Nov 14 01:31:03 GMT [localhost: cf.fsm.takeoverByPartnerDisabled:notice]:Cluster monitor: takeover of by akfiler13 disabled (interconnect error)Wed Nov 14 01:31:03 GMT [localhost: rc:notice]: The system was down for 14secondsWed Nov 14 01:31:03 GMT [localhost:cf.fm.takeoverDetectionSeconds.Default:warning]: optioncf.takeover.detection.seconds is set to 10 seconds which is below the NetAppadvised value of 15 seconds. False takeovers and/or takeovers without diagnosticcore-dumps might occur.Tue Nov 13 16:29:19 AKST [akfiler12: nvram.battery.state:info]: replayed event:The NVRAM battery is currently OFF.Tue Nov 13 16:29:25 AKST [akfiler12: nvram.battery.turned.on:info]: replayedevent: The NVRAM battery is turned ON. It is turned OFF during system shutdown.Tue Nov 13 16:29:27 AKST [akfiler12: cf.nm.nicTransitionUp:info]: replayedevent: Interconnect link 0 is UPTue Nov 13 16:29:27 AKST [akfiler12: iomem.card.enable:info]: replayed event:Caching module in slot 1 has been enabled.Tue Nov 13 16:29:35 AKST [akfiler12: diskown.isEnabled:info]: replayed event:software ownership has been enabled for this systemTue Nov 13 16:29:38 AKST [akfiler12: cf.nm.nicReset:warning]: replayed event:Initiating soft reset on Cluster Interconnect card 0 due to rendezvous resetTue Nov 13 16:29:38 AKST [akfiler12: cf.rv.notConnected:error]: replayed event:Connection for cfo_rv failedTue Nov 13 16:29:38 AKST [akfiler12: cf.nm.nicTransitionDown:warning]: replayedevent: Cluster Interconnect link 0 is DOWN
    • Tue Nov 13 16:29:39 AKST [akfiler12: cf.rv.notConnected:error]: replayed event:Connection for cfo_rv failedTue Nov 13 16:29:39 AKST [akfiler12: sas.link.error:error]: replayed event:Could not recover link on SAS adapter 3c after 10 seconds. Offlining theadapter.Tue Nov 13 16:29:39 AKST [akfiler12: sas.link.error:error]: replayed event:Could not recover link on SAS adapter 3b after 10 seconds. Offlining theadapter.Tue Nov 13 16:29:40 AKST [akfiler12: fmmb.current.lock.disk:info]: replayedevent: Disk 0d.17 is a local HA mailbox disk.Tue Nov 13 16:29:40 AKST [akfiler12: fmmb.current.lock.disk:info]: replayedevent: Disk 0a.45 is a local HA mailbox disk.Tue Nov 13 16:29:40 AKST [akfiler12: fmmb.instStat.change:info]: replayed event:normal mailbox instance on local side.Tue Nov 13 16:29:41 AKST [akfiler12: cf.nm.nicTransitionUp:info]: replayedevent: Interconnect link 0 is UPTue Nov 13 16:29:41 AKST [akfiler12: fmmb.current.lock.disk:info]: replayedevent: Disk 0b.16 is a partner HA mailbox disk.Tue Nov 13 16:29:41 AKST [akfiler12: fmmb.current.lock.disk:info]: replayedevent: Disk 0c.17 is a partner HA mailbox disk.Tue Nov 13 16:29:41 AKST [akfiler12: fmmb.instStat.change:info]: replayed event:normal mailbox instance on partner side.Tue Nov 13 16:29:42 AKST [akfiler12: cf.fm.partner:info]: replayed event:Cluster monitor: partner akfiler13Tue Nov 13 16:29:42 AKST [akfiler12: cf.fm.timeMasterStatus:info]: replayedevent: Acting as cluster time slaveTue Nov 13 16:29:42 AKST [akfiler12: shelf.config.mpha:info]: replayed event:All attached storage on the system is multi-pathed HA.Tue Nov 13 16:30:48 AKST [akfiler12: coredump.spare.none:info]: replayed event:No sparecore disk was found.Tue Nov 13 16:30:48 AKST [akfiler12: raid.cksum.replay.summary:info]: replayedevent: Replayed 0 checksum blocks.Tue Nov 13 16:30:48 AKST [akfiler12: raid.stripe.replay.summary:info]: replayedevent: Replayed 0 stripes.Tue Nov 13 16:30:50 AKST [akfiler12: cf.fm.launch:info]: replayed event:Launching cluster monitorTue Nov 13 16:30:50 AKST [akfiler12: cf.fm.partner:info]: replayed event:Cluster monitor: partner akfiler13Tue Nov 13 16:30:50 AKST [akfiler12: cf.fm.discardNvram:notice]: replayed event:Cluster monitor: node was previously taken over, nvram may be discardedTue Nov 13 16:31:02 AKST [akfiler12: fcp.service.adapter:warning]: replayedevent: FCP started but no adapter is present!Tue Nov 13 16:31:02 AKST [akfiler12: cf.nm.nicReset:warning]: replayed event:Initiating soft reset on Cluster Interconnect card 0 due to rendezvous resetTue Nov 13 16:31:02 AKST [akfiler12: kern.syslogd.error:error]: replayed event:syslog daemon error logerror syslogd: Couldnt find address for host name"lx9239.ak.providence.org": Unknown resolver error .Tue Nov 13 16:31:03 AKST [akfiler12: kern.syslogd.error:error]: replayed event:syslog daemon error logerror syslogd: Couldnt find address for host name"lx9239.ak.providence.org": Unknown resolver error .Tue Nov 13 16:31:03 AKST [akfiler12: kern.syslogd.restarted:info]: replayedevent: syslogd: Restarted.Tue Nov 13 16:31:03 AKST [akfiler12: cf.fsm.takeoverOfPartnerDisabled:notice]:replayed event: Cluster monitor: takeover of akfiler13 disabled (interconnecterror)Tue Nov 13 16:31:03 AKST [akfiler12: cf.fsm.takeoverByPartnerDisabled:notice]:replayed event: Cluster monitor: takeover of by akfiler13 disabled(interconnect error)Tue Nov 13 16:31:03 AKST [akfiler12: kern.syslog.msg:notice]: replayed event:The system was down for 14 secondsTue Nov 13 16:31:03 AKST [akfiler12:cf.fm.takeoverDetectionSeconds.Default:warning]: replayed event: optioncf.takeover.detection.seconds is set to 10 seconds which is below the NetAppadvised value of 15 seconds. False takeovers and/or takeovers without diagnostic
    • core-dumps might occur.Tue Nov 13 16:31:04 AKST [akfiler12: dfu.firmwareUpToDate:info]: Firmware is up-to-date on all disk drivesTue Nov 13 16:31:04 AKST [akfiler12: perf.archive.start:info]: Performancearchiver started. Sampling 23 objects and 211 counters.Tue Nov 13 16:31:06 AKST [akfiler12: cf.fsm.takeoverByPartnerEnabled:notice]:Cluster monitor: takeover of akfiler12 by akfiler13 enabledTue Nov 13 16:31:07 AKST [akfiler12: netif.linkUp:info]: Ethernet e4b: Link up.Tue Nov 13 16:31:07 AKST [akfiler12: netif.linkUp:info]: Ethernet e0a: Link up.Tue Nov 13 16:31:07 AKST [akfiler12: netif.linkUp:info]: Ethernet e0b: Link up.Tue Nov 13 16:31:08 AKST [akfiler12: netif.linkUp:info]: Ethernet e4a: Link up.Tue Nov 13 16:31:08 AKST [akfiler12: pvif.switchLink:warning]: vif1: switchingto e4bTue Nov 13 16:31:09 AKST [akfiler12: rc:ALERT]: timed: time daemon startedTue Nov 13 16:31:10 AKST [akfiler12: cmds.sysconf.logErr:error]: sysconfig:Unless directed by NetApp Global Services volumes vol0, vm01, vm02, vm03, vmpg,ISO, pocvm1, and vm04 should have the volume option create_ucode set to On. .Tue Nov 13 16:31:10 AKST [akfiler12: mgr.boot.disk_done:info]: NetApp Release7.3.6 boot complete. Last disk update written at Tue Nov 13 16:30:44 AKST 2012Tue Nov 13 16:31:10 AKST [akfiler12: cf.hwassist.notifyEnableOn:info]: Clusterhw_assist: hw_assist functionality has been enabled by user.Tue Nov 13 16:31:10 AKST [akfiler12: cf.fsm.takeoverOfPartnerEnabled:notice]:Cluster monitor: takeover of akfiler13 enabledTue Nov 13 16:31:10 AKST [akfiler12: cf.hwassist.intfUnConfigured:info]: Cannotget IP address of preferred e0M ethernet interface for hardware assistfunctionality.Tue Nov 13 16:31:10 AKST [akfiler12: cf.hwassist.intfUnConfigured:info]: Cannotget IP address of preferred e0M ethernet interface for hardware assistfunctionality.Tue Nov 13 16:31:10 AKST [akfiler12: cf.hwassist.hwasstActive:info]: Clusterhw_assist: hw_assist functionality is active on IP address: 10.21.20.57 port:4444Tue Nov 13 16:31:10 AKST [akfiler12: mgr.boot.reason_ok:notice]: System rebootedafter power-on.Tue Nov 13 16:31:10 AKST [akfiler12: cf.hwassist.DefaultPrtnrAddr:notice]: Thesystem automatically chose 10.21.20.58 as the hardware-assist partner address.If you want to use a different IP address, change it using the command optionscf.hw_assist.partner.address.Tue Nov 13 16:31:10 AKST [akfiler12: cf.hwassist.notifyCfgFailed:CRITICAL]:Failed to notify hardware-assist configuration to hardware component (RLM): RLMstatus unknown(4)Tue Nov 13 16:31:10 AKST [akfiler12: rlm.driver.timeout:warning]: The softwaredriver for the Remote LAN Module (RLM) detected a problem: RLM status unknown.Tue Nov 13 16:31:10 AKST [akfiler12: netif.linkDown:info]: Ethernet e4a: Linkdown, check cable.Tue Nov 13 16:31:12 AKST [akfiler12: netif.linkUp:info]: Ethernet e4a: Link up.Tue Nov 13 16:31:20 AKST [akfiler12: console_login_mgr:info]: netapp logged infrom consoleTue Nov 13 16:31:20 AKST [akfiler12: nbt.nbns.registrationComplete:info]: NBT:All CIFS name registrations have completed for the local server.Tue Nov 13 16:31:33 AKST [akfiler12: kern.syslogd.restarted:info]: syslogd:Restarted.Tue Nov 13 16:31:47 AKST [akfiler12: asup.smtp.sent:notice]: ClusterNotification mail sent: Cluster Notification from akfiler12 (SYSTEMCONFIGURATION WARNING) WARNINGTue Nov 13 16:31:47 AKST [akfiler12: asup.smtp.fail:warning]: Autosupport mailto "" failed (Protocol error) for message: REBOOT (power on)Tue Nov 13 16:31:47 AKST [akfiler12: asup.smtp.noRecipients:error]: ShortAutoSupport not sent because there are no valid autosupport.noteto recipientsfor (REBOOT (power on)) using smtplegacy.providence.org.Tue Nov 13 16:31:47 AKST [akfiler12: asup.smtp.drop:error]: Autosupport mail(/etc/log/autosupport/201211131631.1) was not sent for host (0). The system willdrop the messageTue Nov 13 16:32:08 AKST [akfiler12: asup.smtp.sent:notice]: Cluster
    • Notification mail sent: Cluster Notification from akfiler12 (REBOOT (power on))INFOTue Nov 13 16:32:14 AKST [akfiler12: asup.smtp.sent:notice]: ClusterNotification mail sent: Cluster Notification from akfiler12 (PERFORMANCESNAPSHOT) INFOTue Nov 13 16:32:47 AKST [akfiler12: RLM_console_login:info]: p375290 logged infrom RLMakfiler12> exitUse control-D to exitakfiler12> Tue Nov 13 16:48:08 AKST [akfiler12:ses.shelf.reset.capability:info]: Some DS14-Mk4-FC/ESH4 disk shelves on thissystem are capable of automatic power reset, but some do not have appropriatefirmware or PSU model. For improved system RAS, consider upgrading shelves tohave this capability. The environ shelf command shows this features status.akfiler12> Tue Nov 13 16:50:28 AKST [akfiler12: asup.smtp.fail:warning]:Autosupport mail to "" failed (Protocol error) for message: USER_TRIGGERED(2003676616)Tue Nov 13 16:50:28 AKST [akfiler12: asup.smtp.noRecipients:error]: ShortAutoSupport not sent because there are no valid autosupport.noteto recipientsfor (USER_TRIGGERED (2003676616)) using smtplegacy.providence.org.Tue Nov 13 16:50:28 AKST [akfiler12: asup.smtp.drop:error]: Autosupport mail(/etc/log/autosupport/201211131650.0) was not sent for host (0). The system willdrop the messageTue Nov 13 16:50:47 AKST [akfiler12: asup.smtp.sent:notice]: ClusterNotification mail sent: Cluster Notification from akfiler12 (USER_TRIGGERED(2003676616)) INFOakfiler12> Tue Nov 13 17:00:00 AKST [akfiler12: kern.uptime.filer:info]:5:00pm up 30 mins, 1066863 NFS ops, 174 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSIopsakfiler12>akfiler12> Tue Nov 13 17:37:01 AKST [akfiler12:raid.tetris.media.recommend.reassign.err:info]: Block recommended forreassignment on Disk /aggr0/plex0/rg0/0d.24 Shelf 1 Bay 8 [NETAPPX279_HVPBP288F15 NA02] S/N [JLWDVURC], block #57205352 during stripe writeTue Nov 13 18:00:00 AKST [akfiler12: kern.uptime.filer:info]: 6:00pm up 1:303053608 NFS ops, 497 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI opsakfiler12> Tue Nov 13 19:00:00 AKST [akfiler12: kern.uptime.filer:info]:7:00pm up 2:30 5048051 NFS ops, 790 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSIopsTue Nov 13 19:00:30 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of hourly snapshot (previous auto-delete snap create still inprogress)Tue Nov 13 19:02:19 AKST [akfiler12: openssh.invalid.channel.req:warning]: SSHclient (SSH-2.0-OpenSSH_4.3) from 10.21.27.118 sent unsupported channel request(10, env).license ?license usage: license license add <code> ... license delete <service> ... license helpakfiler12> license a_sis GKXXDCG cifs CNDWHEM cluster CHSTIVH cluster_remote not licensed compression not licensed disk_sanitization not licensed fcp MUZKSHL flash_cache not licensed
    • flex_clone not licensed flex_scale KOGYDCG flexcache_nfs not licensed gateway_hitachi not licensed http not licensed iscsi OFMEYRI multistore not licensed nearstore_option CEVTIVH nfs EWQGUPC operations_manager not licensed protection_manager not licensed provisioning_manager not licensed smdomino not licensed smsql not licensed snapdrive_unix not licensed snapdrive_windows not licensed snaplock not licensed snaplock_enterprise not licensed snapmanager_hyperv not licensed snapmanager_oracle not licensed snapmanager_sap not licensedsnapmanager_sharepoint not licensed snapmanagerexchange not licensed snapmirror EWELSHL snapmirror_sync not licensed snapmover not licensed snaprestore CJBWETB snapvalidator not licensed sv_applications_pri not licensed sv_exchange_pri not licensed sv_linux_pri not licensed sv_marketing_pri not licensed sv_ontap_pri not licensed sv_ontap_sec not licensed sv_oracle_pri not licensed sv_sharepoint_pri not licensed sv_sql_pri not licensed sv_unix_pri not licensed sv_vi_pri not licensed sv_vmware_pri not licensed sv_windows_ofm_pri not licensed sv_windows_pri not licensed syncmirror_local not licensed v-series not licensed vld not licensedakfiler12> Tue Nov 13 20:00:00 AKST [akfiler12: kern.uptime.filer:info]:8:00pm up 3:30 10798686 NFS ops, 1034 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSIopsakfiler12> Tue Nov 13 21:00:00 AKST [akfiler12: kern.uptime.filer:info]:9:00pm up 4:30 15034062 NFS ops, 1338 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSIopsakfiler12> Tue Nov 13 22:00:00 AKST [akfiler12: kern.uptime.filer:info]:10:00pm up 5:30 17015199 NFS ops, 1590 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSIopsTue Nov 13 23:00:00 AKST [akfiler12: kern.uptime.filer:info]: 11:00pm up 6:3020000256 NFS ops, 1873 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI opsWed Nov 14 00:00:00 AKST [akfiler12: kern.uptime.filer:info]: 12:00am up 7:3022786647 NFS ops, 2168 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI opsWed Nov 14 00:00:16 AKST [akfiler12: wafl.snap.create.skip.other:notice]: aggr0:skipping creation of nightly snapshot (previous auto-delete snap create still inprogress)Wed Nov 14 01:00:00 AKST [akfiler12: kern.uptime.filer:info]: 1:00am up 8:30
    • 29656809 NFS ops, 2482 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops