by admin

Add License Key - Smart Array P410i In Embedded Slot

HomeSwitchboardUnix AdministrationRed HatTCP/IP NetworksNeoliberalismToxic Managers
May the source be with you, but remember the KISS principle ;-)
Skepticism and critical thinking is not panacea, but can help to understand the world better
  1. Add License Key - Smart Array P410i In Embedded Slot Machine
  2. Add License Key - Smart Array P410i In Embedded Slots
  3. Add License Key - Smart Array P410i In Embedded Slot Machines
  4. Add License Key - Smart Array P410i In Embedded Slot Software
NewsHP ServersRecommended LinksHP ProLiant DL360 G7Recovering Filesystems from corrupted RAID setsAbrupt change of disks geometry in RAID10 configurationLack of reliability
Sysadmin Horror StoriesAdministration of Remote ServersLinux Disk PartitioningSimple Unix Backup Tools Recovery of LVM partitionsHumor Etc
  • Overcomplexity and horrible reliability in non-standard situations

Introduction

Solved: Hi all, I have paid a Smart Array Advanced Pack from HP reseller. After installing the license to P410 controller, i see the license information ( picture ). HP Smart Array Advanced Pack 8 HP Smart Array Advanced Pack About SAAP SAAP is a collection of additional and advanced controller features embedded in the firmware of select Smart Array controllers. When activated with a registered license key, SAAP 1.0 provides the following features:. RAID 6 (ADG). RAID 60. Advanced Capacity Expansion.

This controller is used in HP DL360 G7 DP580 G7 and several other HP servers. Firmware version vary and in a typical HP manner are updated way too frequently. You need to disable graphic boot screen to get into controller BIOS on startup

  • Press the F9 key when prompted during POST to enter BIOS
    • Select 'Advanced Options'
    • Select 'Advanced System ROM options'
    • Select Power-on Logo
    • Disable it

With CD/DVD installed on HP DL360 G7 P410i controller does not see drives in slots 5 and 6 (Bay 5, 6 not available when DVD installed) and sometimes even in slots 1 and 2 (in version 5.11). Only drives in slots 3 and 4 are visible.

‎07-06-2012 03:20 PM

I just added 2 new drives to my 2 DL360G7 servers which are the same build and config. I am not seeing the system recognize them at all with the SmartStart ACU or even physically, no lights or anything are apparent when the drives are insterted into the system. They are the same configuration, including same part number and all are 15k. I feel that I might have missed something, but dont' think I need another controller to use these slots, would someone please give me an idea if there is something else I am missing?

The system is currently configuration o 2x146GB sets of Raid 1+-0. I just expanded the original array from 2 to 4 drives, but the 2 new drives which would make this 6 in the system aren't visible at all.

Thanks in advance for your help.

Re: DL360 G7 not showing new drives in slots 5 & 6

‎07-08-2012 07:20 PM

Take a look at the below links:

DL360 G7 QuickSpecs.

DL360 G7 Storage Configuration.

I am not sure if you have 8 HDD backplane or 6 HDD backplane with DVD Drive.

Bay 5, 6 not available when DVD installed.

4 SFF Bays standard with option for up to 8 SFF drive bays total.
NOTE: 4 bays standard in Entry, Base and CTO models, with optional upgrade to 8 bays with the SFF HD Backplane Kit (516966-B21). No CD or DVD options available when server is configured with 8 drive bays.
Thanks

Re: DL360 G7 not showing new drives in slots 5 & 6

‎07-09-2012 08:16 AM

I had in production originally 4 146GB SAS drives and DVD drive which is in slot 7 and 8, but 5 and 6 just had an insert.

So, you are saying because of having the DVD drive, slots 5 and 6 are only capable of an insert? Or, do I need another component to use this space? Thanks.

Suman_1978 HP Pro

Re: DL360 G7 not showing new drives in slots 5 & 6

‎07-09-2012 07:18 PM

The documentation says that if Optical drive is installed then only drives bay 1-4 works, 5-6 do not work.

You may also refer to this link:

HP ProLiant DL360 G7 Server User Guide
http://bizsupport1.austin.hp.com/bc/docs/support/S​upportManual/c02065265/c02065265.pdf
Page# 48 = This server supports the installation of a DVD-ROM drive or a DVD-RW drive. When an optical drive is installed, the server does not support the additional hard drive backplane.
Page# 51 = When the hard drive backplane option is installed, the server does not support the DVD-ROM or DVD-RW drive options.
Page# 68 and 69 = Hard drive backplane cabling

Thank You!
__________________________________________
Was the post useful? Click on the white KUDOS! Star.
For product specific Guided Troubleshooting visit; http://www.hp.com/support/hpgt
Want to know How to use HP Guided Troubleshooting? Visit the below link.
http://h20000.www2.hp.com/bizsupport/TechSupport/D​ocument.jsp?objectID=c03283214

RAID10 on two disks is fools gold

The small amount of performance enhancement that is achievable by 'pseudo RAID10' on two disks is negated by loss of reliability: if with RAID 1 disks are usable even if controller failed or lost configuration, disks configured with RAID 10 are not. Moreover sometimes very bad things can happen with RAID 10 configuration on to disks, especially with ancient version of firmware on P410i (5.xx or earlier): controller can lose both disks at one by issuing some internal command that wipes that content.

If you do not touch controller and disks they work. But if you do touch them your mileage can vary.

First of all HP in its infinite wisdom propose the most aggressive RAID setting. They advocate striping and propose 51 on two disks by default. But with cache of 256M the effect is negligible and with 1GB cache might be negative. But if RAID 1 contains disks that can be read on other server individually, RAID 10 is more complex beast and loss of configuration in controller lead to real disasters.

All RAID levels which offer striping can get into situation when all of the data on the array are lost when more hard drives fail than the redundancy can handle. Or when the controller loses it s RAID configuration. Which is probably most frequent case with P410i.

Setting up RAID 10 on 2 disks involves additional magic with 'Sections' on the drive to speed up sequential reads. The gain in unclear and can be negative(Non-standard RAID levels - Wikipedia). Generally Raid 10 requires 4 disks with non-standard implementation existing for three (Nested RAID levels - Wikipedia) ?

RAID 10, as recognized by the storage industry association and as generally implemented by RAID controllers, is a RAID 0 array of mirrors, which may be two- or three-way mirrors,[6] and requires a minimum of four drives. However, a nonstandard definition of 'RAID 10' was created for the Linux MD driver;[7] Linux 'RAID 10' can be implemented with as few as two disks. Implementations supporting two disks such as Linux RAID 10 offer a choice of layouts.[7] Arrays of more than four disks are also possible.

Or RAID 50:

RAID 50, also called RAID 5+0, combines the straight block-level striping of RAID 0 with the distributed parity of RAID 5.[3] As a RAID 0 array striped across RAID 5 elements, minimal RAID 50 configuration requires six drives. On the right is an example where three collections of 120 GB RAID 5s are striped together to make 720 GB of total storage space.

One drive from each of the RAID 5 sets could fail without loss of data; for example, a RAID 50 configuration including three RAID 5 sets can only tolerate three maximum potential drive failures. Because the reliability of the system depends on quick replacement of the bad drive so the array can rebuild, it is common to include hot spares that can immediately start rebuilding the array upon failure. However, this does not address the issue that the array is put under maximum strain reading every bit to rebuild the array at the time when it is most vulnerable.[12][13]

RAID 50 improves upon the performance of RAID 5 particularly during writes, and provides better fault tolerance than a single RAID level does. This level is recommended for applications that require high fault tolerance, capacity and random access performance. As the number of drives in a RAID set increases, and the capacity of the drives increase, this impacts the fault-recovery time correspondingly as the interval for rebuilding the RAID set increases.[

How you can recover information is it is distributed on two disks. But novice sysadmin do not ask such questions and accept proposed default.

Generally using some advanced version of RAID without testing the controller capabilities (and reliability) is very dangerous. Only RAID 1 allow space for errors in this respect. Recovery of 300GB disk can easily cost $15K

Overcomplexity and horrible reliability in non-standard situations

There is no documentation where and how the controller stores this configuration. If configuration is lost you lost all (and I mean all) information on your disks. With Raid 10 you need to pay special recovery company to retrieve the data even if you disk are not damaged at all. The price for recovery of 300GB in such a situation is between $5K and $15K. See for exampleRAID 10 Data Recovery - Secure Data Recovery Services Essentially the cost of the server. A controller bug in which it lose the configuration can do as much damage as a failed disk.

Moreover, HP does not provide qualified advice if you face such a situation. One remedy is to buy their recovery contract. That increases the cost.

Among non-standard situation for this RAID controller

  • Removing the drive (even not related to RAID) and immeduatly after that shutting the system down
  • Shutting down the serve while it rebuilds on the drives.

In some, yet unclear, circumstances the controller can issue formatting command for the drives. Disks will continue to be shown as RAID 10 and healthy but there no readable information on them.

Case study: total loss of data on the server

Here is one interesting case study:

We have problem with HP DL360 server that has four 300GB 15 RPM SAS drives (standard HP suppled drive that came with the server, I think they are Seagate. All of the same model and batch; no replacements). Two were used in RAID 10 configuration (in slots 3 and 4 of the server) -- DL360 in our configuration has 6 slots and DVD in slots 7 and 8. Two were unused.

Drives are controlled to P410i controller. Firmware Version: 5.14.

Again, the server has four 300GB 15K RPM drives (slots 1-4), but only two were used (slots 3 and 4) as the second pair was not visible in the controller BIOS interface

I wanted to add two additional drives as we run out of space on this server, but as the drives in slots 1 and 2 were not visible in controller BIOS interface (as I lately discovered from Internet this is a known problem with P410i) I decided to move them the drives from slot 1 and 2 to slot 5 and 6 without shutting the server down. Which proved top be a very bad idea that I believe caused this problem.

I noticed that something was wrong when I removed the drive as controller complained about some data lost, so it looks like despite the fact that they were not visible via controller interface controller saw them and may be cached some portion of the data on those 'unused' drives.

As the result OS can't boot and my yesterday inspection of harddrive via DD utility after booting from rescue DVD shows zeros in the initial sectors. Looks like controller forcefully reinitialised RAID10 with predictable consequences for data. Which can't be right action.

RAID is operational and drives are perfectly readable from the rescue DVD using DD. No errors reported. DD copied as many sectors as I asked without complains. But no information on the drives exist anymore.

HP Smart Array - When is Magic Just an Illusion?

Reproduced verbatim from HP Smart Array - When is Magic Just an Illusion by Eric Wu

I would definitely recommend a backup prior to expanding an HP array. Interrupting the process is almost guaranteed to cause data loss it seems.

Introduction

While testing new HP G7 servers meant for database applications at work, the category of storage is an important one. Certain function such as online RAID expansion and migration allows one to reduce the initial cost and purchase storage as required.

I've been hearing and for the most part, experiencing what a few has described as HP Magic. Simply put, complicated things just work, we don't know how they get it to work so well, but it does. There are times where this is not true, and in testing of the Smart Array P410i on the DL360 G7 servers, I realized how unsafe it is to expand arrays on these and possibly other controllers.

HP ensures that a battery backup module is installed before allowing array expansion. This creates the idea that the expansion process may recover in the event of loss of power, hard shutdown, reboot, etc. This is not documented however, resulting in my curiosity to test how well it works.

Test 1 - Reboot

  • Array transformation progress at over 10%
  • Data resides within a 30GB root partition
  • 4 x 500GB SATA drives in RAID 0+1
  • Adding 2 x 500GB SATA drives
  • Drive write cache disabled
  • Oracle Linux 5.6 x86_64
  • Using hpacucli for array management
  • Server rebooted from SSH session

Test 1 Result


Looking at the last few lines of the output, seems like this controller has failed to resume. I was able to boot into the OS after pressing F2. However, all it means is that the 10% completion contains at least enough of the data required to boot Linux.

Test 2 - Power Unplugged

  • Array transformation progress at only 1%
  • Data resides within a 30GB root partition
  • 4 x 500GB SATA drives in RAID0+1
  • Adding 2 x 500GB SATA drives
  • Drive write cache disabled
  • Oracle Linux 5.4 x86_64 (did not bother updating this time)
  • Using hpacucli for array management
  • Unplugging both PSUs
Test 2 Results

Same error as previous test. The difference is that in this case, only 1% of the transformation has completed, resulting in an MBR corruption upon pressing F2. The following screen is displayed on most HP servers when this happens.

Further Analysis and Research

While searching for information on this issue, I came across an HP troubleshooting document. It outlined the error displayed in Test 2 Results above (1769 - page 150). The suggested course of action is:

  • Press the F2 key to accept the data loss and re-enable the logical drives
  • Restore data from backup
  • Replace drive or array accelerator (cache module) as appropriate
Link: HP Proliant Servers Troubleshooting Guide

HP's product manual for Smart Array P400 or P410 did not mention any ability to resume from any of the above scenarios. It also stated that removal of a drive from an array during expansion or migration would result in data loss! This is something I will need to test tomorrow, as it is a more likely case, next to software related failures.

What about offline expansion? The answer is: Not Possible. HP did not provide a means to manage the array within the Option ROM (RAID ROM) for functions such as expansion, migration, or any of the other settings. I personally have an Areca ARC-1230, which provides me with nearly all functions available in CLI and the dedicated Ethernet connection for HTTP based management.

Conclusion

While well known companies make it seem like things work well, like magic. What we know about magic is that, often times, it's just an illusion. Make sure to do testing prior to going into production environment in order to determine risky use cases and corresponding course of action for each failure scenario.

8 comments

Comment from: wb [Visitor]

I'm currently doing an offline transformation using the array configuration utility on the smartstart cdrom. I was wondering if I could reboot the server into linux. But after reading your blog I've decided to wait till its finished :)

06/07/11 @ 05:56

Comment from: TooMeeK [Visitor]

For me, it look's like controller firmware bug.

This shouldn't happend. Try upgrading firmware, for ML350 I remember that all diodes of drives flash to blue when upgrading firmware of the RAID controller.
'It also stated that removal of a drive from an array during expansion or migration would result in data loss!' ee.. what? so this means that if You put brand new drive into array and it will fail this can cause data loss?

Comparing this to software Linux raid using mdadm: when drive fails during rebuild it become failed and rebuild stops. System will stay alive. But I recently noticed that SATA onboard set as IDE not AHCI can block controller and system will hang if drive will fail.

Also on older servers with SCSI drives if drive fail, but only one partition will be degraded and other are still used this can hand system until reboot. So setting manually all partitions from broken drive as failed prevent hungs.

08/03/11 @ 14:58

Comment from: Eric Wu [Member]

It's already at the latest firmware. I've never seen something like this before. This might be related to expansion only. I'd like to get my hands on some other controllers to see how well they do in this area. Rebuilding is usually not an issue as the content on existing drives are not affected. Expansion may modify current drive data which results in this issue. However, logically speaking, the redundancy is supposed to take care of a drive failure if the expansion algorithm is properly designed and the cache backup is meant to protect from power outages or reboots when expanding the volume.

08/03/11 @ 15:32

Comment from: Clarence [Visitor]

I definitely have the same opinion with you concerning this topic. Nice entry. Already bookmarked for future reference. 00-)

11/04/11 @ 08:54

Comment from: kaitoollayli1982 [Visitor]

I am impressed, I must say. Really rarely do I discovered a blog thats both educative and entertaining, and let me tell you, you have hit the nail on the head. Your thoughts is important; the issue is something that not enough people are speaking intelligently about. I am very happy that I stumbled across this in my search for something relating to it. =-=

12/19/11 @ 19:02

Comment from: Troy_J [Visitor]

I just ran across a similar scenario while running the off-line ACU utility to expand an array. Added two more 600GB SAS drives to a 4-drive array. The Extension phase went through the night for several hours. I didn't need to migrate (settings stayed the same), but the progress bar for the Expansion phase stayed at 0% for about an hour. That's when an exec called me and insisted that he NEEDED data on that server IMMEDIATELY. Scheduled downtime or not. I couldn't find any reference to interrupting or cancelling the process, especially for the off-line tool. Calculations of 15mins/GB had me wondering if this thing would take the entire weekend to complete. I even called HP out of desperation and was told that their tests weren't conclusive. 'Sometimes the data got lost. Sometimes it didn't. So we can't guarantee anything.' While I was thinking of a way around this, I hit the 'ReScan' button for something to do... and the progress meter showed 72%! 20 minutes later, everything was complete. (Odd that the progress meter worked in real-time while extending.)
So I wasn't able to conclusively test this, I'm kind of glad I'm not stuck doing hours of data restoration over a weekend.

05/18/13 @ 15:12

Comment from: Eric Wu [Member]

Glad it worked out for you Troy. I would definitely recommend a backup prior to expanding an HP array. Interrupting the process is almost guaranteed to cause data loss it seems.

Add License Key - Smart Array P410i In Embedded Slot Machine

05/28/13 @ 02:58

Comment from: Tiago de Aviz [Visitor]

Hello Eric,
Another case happened on a customer of mine where the main drive failed on a RAID 1 group, and we had data loss. It seems as if the server never actually 'switched' to the secondary disk...
The more odd thing is that the mirror on the second drive appeared to be outdated, missing files created 14+ days ago.
I'm staying away from these controllers.
Regards,
Tiago

06/29/13 @ 16:50

NEWS CONTENTS

  • 200102 : 1783 - Slot 0 Drive Array Controller Failure - p41... - HP Enterprise Business Community ( 1783 - Slot 0 Drive Array Controller Failure - p41... - HP Enterprise Business Community, )
  • 200102 : raid - data lost with RAID5 on proliant DL360 when drives fail ( raid - data lost with RAID5 on proliant DL360 when drives fail, )
  • 200102 : Server Fault ( Server Fault, )

Old News ;-)

Top Visited
Switchboard
Latest
Past week
Past month

1783 - Slot 0 Drive Array Controller Failure - p41... - HP Enterprise Business Community

Post options
  • Mark as New
  • Bookmark
  • Subscribe
  • Email to a Friend

‎01-04-2012 06:52 AM

We have had these servers deployed since mid-March with no problems until we started updating the firmware and BIOS in late October. Since then, we have averaged about 1 server failure every 6 weeks with this error. HP tries to insist that it is the controller's fault. We have found through troubleshooting that if we install known good drives into the box, the problem goes away. If we install the failed drives into another known good box, this error will follow the drives, not the controller. The drives (both of them) are unusable at this point.

We can remove the drives from the box, run SmartStart utility and the controller is visible. If we reinsert the drives and run SmartStart, the controller is not visible.

The following message is displayed during POST, it appears to be very specific with the dlu code, but I haven't been able to find any kind of information:

1783 – Slot 0 Drive Array Controller Failure! Command failure (cmd=0h, err=00h, dlu=013:5h)

System configuration:

ProLiant ML350 G6 (2011.05.05 – D22)
Smart Array p410i (5.06) (RAID 0+1)
2 x 300GB SAS drives(various, EF0300FATFD – HPD2, EF0300FARMU – HPD3)
Windows Server 2008 (W32 – Standard) – SP2

1 person had this problem.
  • 1783
  • dlu
View All (3)

Occasional Advisor

Posts: 8

Registered: ‎03-24-2003

Message 2 of 5 (24,442 Views)

Re: 1783 - Slot 0 Drive Array Controller Failure - p410i

Post options
  • Mark as New
  • Bookmark
  • Subscribe
  • Email to a Friend

‎01-04-2012 09:38 AM

Have you run the firmware upgrade (version 9.3) to update the FW on the drives? make sure they have updated firmware too. There has been quite a bit of chatter on drive FW incompatibility with the new P series of controllers (which are not HP, but just branded HP).

You have to put the drivers into the box and then boot with the FW DVD to update the FW on the drive. The FW update manager will automatically update the FW on the drives.

I think you have FW HPD2 and HPD3 on your drives. Not sure where the current revision is for your type of drive.

Have you tried a new controller?

vroc31426

Occasional Collector

Posts: 2

Registered: ‎11-05-2010

Message 3 of 5 (24,439 Views)

Re: 1783 - Slot 0 Drive Array Controller Failure - p410i

‎01-04-2012 12:06 PM

Thanks for the info on the controller, I will look into it. Back when all of this started, we were pushing the drive f/w updates through HPSIM. That was when we had our first occurance. The controller is integrated on the system board, no cache module, but when we move drives to another system, problem follows the drives.. Known good drives work fine.

raid - data lost with RAID5 on proliant DL360 when drives fail

Server Fault

up vote0down votefavoriteFor some reason the Array Controller on my Proliant DL360 G6 couldn't recognize 2 of the 6 750GB drives I am running in a RAID 5 configuration with VMware ESXi 5.1.

When I rebooted the server I chose the BIOS option (F2) to recognize the 2 drives it said it stopped acknowledging.

Here is the BIOS option I chose: 'Select 'F2' to accept data loss and re-enable logical drive(s).'

All 6 drives now show up in the volume again.

Unfortunately, there seems to be data corruption. Many of the virtual machines no longer work and don't even register properly in ESXi anymore. ESXi boots ok, but none of the virtual machines hosted in it work.

I booted with the Array Configuration Utility and it says the Parity Initialization has finished. ACU doesn't show any other errors or information notices.

Is there a way for me to rebuild or recover my data so my virtual machines start working again? It is still a mystery to me why Array stopped seeing the two drives in the first place, but all I want to do now is recover all my data so my virtual machines start working again.

raid hp hp-proliant>edited Jun 27 '13 at 11:53ewwhite
130k29223470 		asked Jun 26 '13 at 23:52 user127875
412
5
'Select 'F2' to accept data loss and re-enable logical drive(s).' ...uh... – Nathan C Jun 27 '13 at 1:32
6
Restore backups. – fukawi2 Jun 27 '13 at 1:40
add a comment

2 Answers 2

Add License Key - Smart Array P410i In Embedded Slots

activeoldestvotes
up vote7down voteF2 is typically the right option to choose... Otherwise the ESXi server would not have booted. My concern is what happened leading up to this incident...

Did you receive any errors? Any indicators on the hard drive LEDs? Typically, an HP server's disks won't just crap-out on you. Considering you're using 750GB disks in RAID5, the chances are that the drives are SATA and you may have more than one failed or failing disk.

Let's go the the HP ProLiant DL360 G6 quickspecs...

Okay, so the only disk options for that server from HP are:

  • SAS 2.5' in 72GB, 146GB, 300GB, 450GB, 600GB...
  • SATA 2.5' in 120GB, 160GB, 250GB, 500GB, 1TB...

So, where did these disks come from?
They're definitely not HP disks. I don't recall any server-class 2.5' 750GB disks ever hitting the market.

Are these laptop hard drives?

If so, there are a number of reasons this could have happened. I think a big SATA RAID5 could have resulted in the dreaded unrecoverable-read error (URE), where you may have had a failed disk and another one on it's way out.

Since this is ESXi, let's hope you have the HP health agents and utility bundle installed.

If you do, post the screenshot of the Hardware Status -> Storage menu in VMware and possibly the output of the /opt/hp/hpacucli/bin/hpacucli ctrl all show config detail command.

Worst case, your data is hosed.

Best case, you can build new virtual machines and import the VMDK files. Maybe it's just .vmx file corruption.

Either way, you should not move forward until you determine what happened with your disks in the first place. Otherwise, you're building on a pile of s**t and could encounter the same thing in the future.

(also, update your server's firmware, if you haven't already)

share improve this answeredited Jun 27 '13 at 13:42
answered Jun 27 '13 at 11:36 ewwhite
130k29223470
Thanks for the great response! Yes, I am using laptop hard drives. I hadn't installed the HP health agents/utility bundle, but I will do so this time. I am convinced I just have to re-install ESXi and restore my backups. My investigations into RAID 5 show that if you lose more than 1 drive at a time (no matter how many drives in the RAID) you won't be able to automatically recover. I do have backups, so all is not lost. It will just take me a day of working in the noisy datacenter to recover. :( The firmware is completely up to date. – user127875 Jun 27 '13 at 14:59
2
NO LAPTOP HARD DRIVES IN THE SERVER!! Don't reinstall onto the same disks!! – ewwhite Jun 27 '13 at 15:42
Your comment 'RAID 5 show that if you lose more than 1 drive at a time (no matter how many drives in the RAID) you won't be able to automatically recover.' isn't necessarily true, depending on what failed. Sometimes the system will mark multiple drives as bad but you might be able to mark them as good/online and see if the system boots. At that point it's definitely time to take a backup and then investigate (via software/TAC) why the system thought the drive(s) were bad. I've had this happen on multiple servers simply due to a bug in the RAID firmware. – TheCleaner Jun 27 '13 at 15:50
add a comment

Add License Key - Smart Array P410i In Embedded Slot Machines

up vote3down voteIf it has rebuilt the array and you still cant access the data ('I booted with the Array Configuration Utility and it says the Parity Initialization has finished. ')

It is time to seek professional help. If it has rebuilt only using 5 drives then it may take some serious detangling to get anything back.

Depending where you are look for a reputable data recovery company.

Mike

share improve this answeranswered Jun 27 '13 at 10:46 Mike
311
2
And a lot of money. Possibly / preferably a job outside IT - one does not require reading (i.e. ignoring 'accept data loss' and then wondering why the data is gone). – TomTom Jun 27 '13 at 11:05
add a comment

[Aug 02, 2015] HP DL360 G7 P410i controller troubleshooting

up vote1down votefavorite

Server is HP DL360 G7 with P410i disk controller. 2xE5620 CPU's. 16GB RAM. Linux mysql 2.6.32-5-amd64 #1 SMP Mon Feb 25 00:26:11 UTC 2013 x86_64 GNU/Linux (Debian 6.0.7)
hpacucli 'ctrl all show status'

hpacucli 'ctrl all show config'

hpacucli 'ctrl slot=0 ld all show'

I run the following script via night:

/root/ubuntu.iso size is abou 2 GB.

in syslog has some errors. I think that it is related to disk controller:

There are no other error messages.

Or this error can be related to memory? I already run memtest86+ on that server for several days and there was no errors.

When server was in data center, i cant boot server up. It show all the time error:

After transporting it to my work, it boot up normally. In ILO event log has fallowing errors:

I already updated bios, disk controller and drive firmwares to latest versions.

HP SmartArray P400: How to repair failed logical drive?

I have a HP Server with SmartArray P400 controller (incl. 256 MB Cache/Battery Backup) with a logicaldrive with replaced failed physicaldrive that does not rebuild.

This is how it looked when I detected the error:

I thought that I had drive 2I:1:8 configured as a spare for Array A and Array B, but it seems this was not the case :-(. I noticed the problem due to I/O errors on the host, even if only 1 physicaldrive of the RAID5 is failed.

Does someone know why this could happen? The logicaldrive should go into 'Degraded' mode but still be fully accessible from the host os!?

I first tried to add the unassigned drive 2I:1:8 as a spare to logicaldrive 2, but this was not possible:

Interestingly it is possible to add the unassigned drive to the first array without problems. I thought maybe the controller put the array into 'failed' state due to the missing spare and protects failed arrays from modification. So I tried was to reenable the logicaldrive (to add the spare afterwards):

But as you can see, re-enabling the logicaldrive this was not possible.

Now I replaced the failed drive by hotswapping it with the unassigned drive. The status now looks like this:

The logical drive is still not accessible. Why is it not rebuilding?

What can I do?

FYI, this is the configuration of my controller:

Thanks for you help in advance.

Recommended Links

Google matched content

Softpanorama Recommended

Top articles

Sites

Top articles

Sites

...

HP Smart Array - When is Magic Just an Illusion by Eric Wu

Add License Key - Smart Array P410i In Embedded Slot Software

    [PDF] RAID 1(+0): breaking mirrors and rebuilding drives

    Dec 15, 2013

    Etc

    Society

    Groupthink :Two Party System as Polyarchy : Corruption of Regulators :Bureaucracies :Understanding Micromanagers and Control Freaks : Toxic Managers : Harvard Mafia :Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience :Who Rules America :Neoliberalism : The Iron Law of Oligarchy : Libertarian Philosophy

    Quotes

    War and Peace: Skeptical Finance : John Kenneth Galbraith :Talleyrand :Oscar Wilde :Otto Von Bismarck :Keynes :George Carlin :Skeptics :Propaganda : SE quotes : Language Design and Programming Quotes :Random IT-related quotes : Somerset Maugham :Marcus Aurelius :Kurt Vonnegut :Eric Hoffer :Winston Churchill :Napoleon Bonaparte :Ambrose Bierce : Bernard Shaw : Mark Twain Quotes

    Bulletin:

    Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis :Political Skeptic Bulletin, 2013 :Unemployment Bulletin, 2010 : Vol 23, No.10 (October, 2011) An observation about corporate security departments :Slightly Skeptical Euromaydan Chronicles, June 2014 :Greenspan legacy bulletin, 2008 :Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) :Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 :Inequality Bulletin, 2009 :Financial Humor Bulletin, 2008 :Copyleft Problems Bulletin, 2004 :Financial Humor Bulletin, 2011 :Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult :Political Skeptic Bulletin, 2011 :Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

    History:

    Fifty glorious years (1950-2000): the triumph of the US computer engineering :Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds :Larry Wall :John K. Ousterhout : CTSS : Multix OSUnix History : Unix shell history :VI editor :History of pipes concept :Solaris : MS DOS : Programming Languages History :PL/1 : Simula 67 :C :History of GCC development : Scripting Languages :Perl history :OS History : Mail :DNS : SSH : CPU Instruction Sets :SPARC systems 1987-2006 :Norton Commander :Norton Utilities :Norton Ghost :Frontpage history :Malware Defense History :GNU Screen : OSS early history

    Classic books:

    The Peter Principle : Parkinson Law : 1984 :The Mythical Man-Month : How to Solve It by George Polya :The Art of Computer Programming :The Elements of Programming Style :The Unix Hater’s Handbook :The Jargon file :The True Believer :Programming Pearls :The Good Soldier Svejk : The Power Elite

    Most popular humor pages:

    Manifest of the Softpanorama IT Slacker Society :Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story :The Cuckoo's Egg :IT Slang : C++ Humor : ARE YOU A BBS ADDICT? :The Perl Purity Test :Object oriented programmers of all nations : Financial Humor :Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor :Goldman Sachs related humor :Greenspan humor : C Humor :Scripting Humor :Real Programmers Humor :Web Humor : GPL-related Humor : OFM Humor :Politically Incorrect Humor :IDS Humor : 'Linux Sucks' Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor :Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor :Networking Humor :Shell Humor :Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 :Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor :Education Humor : IBM Humor : Assembler-related Humor :VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

    The Last but not LeastTechnology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

    Copyright © 1996-2020 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

    FAIR USE NOTICEThis site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

    This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

    You can use PayPal to to buy a cup of coffee for authors of this site

    Disclaimer:

    The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society.We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

    Last modified: March, 12, 2019

Contents

  • 2 Usage

HP provides a command line utility called HPE Smart Storage Administrator (HPE SSACLI) used to configure, manage and monitor their RAID storage controllers. This application has changed its name over the years but is now called ssacli.

Previous names were:

hpacucli –> hpssacli –> ssacli

Fortunately HP provide an installation script which can be found here. To install:

You’re likely to get an error message such as the following:

If so, issue the below command to add the public key:

Now issue apt-get update again.

Now you should be able to install ssacli.

You can confirm this is working with:

Rescan for New Devices

Show Configuration

Show Controller Status

If you have multiple controllers, replace all with the slot you are interested in viewing.

Show Detailed Controller Information

If you have multiple controllers, replace all with the slot you are interested in viewing.

Show Physical Disk Status

0

Show Detailed Physical Disk Information

Show Logical Drive Status

Show Detailed Logical Drive Status

Create New RAID-0 Logical Drive

Create New RAID-1 Logical Drive

Create New RAID-5 Logical Drive

License

Delete Logical Drive

Add New Physical Drive to Logical Volume

Add Spare Disks

Enable Drive Write Cache

Disable Drive Write Cache

Delete Physical Drive

Turn On Blinking Physical Disk LED

Turn Off Blinking Physical Disk LED

Modify Smart Array Cache Read/Write Radio

The cache ratio is determined by read ratio divided by write ratio.

Enable Smart Array Write Cache

Disable Smart Array Cache for Certain Logical Volume

Enable Smart Array Cache for Certain Logical Volume

Enable SSD Smart Path

Disable SSD Smart Path

Resources