Nobody has ever said that FusionIO cards were slow (because they aren’t). Especially if you compare their performance to regular spinning media, or high performance SANs. After all, no SAN will allow you to measure storage write latency in microseconds.

Anyone that has had their database reside on a FusionIO card has had nothing but good things to say about them. The only thing is that a lot of people are probably not making the most out of the cards that they have. FusionIO cards actually have an artificial performance restriction put on them, and can go even faster than they do now.

Why Would My Card Be Slow?

I wouldn’t say your card was slow, but there is potential that you could be getting a lot more from it than you do now.

When shipped and installed all of the cards have a restriction around the amount of power that they can draw from the PCIe slot in which they are mounted. This is set to a nice safe 24.5 Watt value, which is just a touch less than the PCIe Gen2 x8 slots would provide (that standard being 25W).

Some hardware manufacturers increased the power output in those slots, however, (unless reported by the system BIOS) the cards will not draw additional power, which can lead to performance throttling.

With PCIe Gen3 x16 slots many hardware manufacturers provide 75W to the slot (and some, like the HP DL380 G8 up to 150W with optional power cords).  The FusionIO cards are not installed (by default) ready to take advantage of the extra power available and so you might not be getting all the performance that you could from the card.

 

How Do I Know If I’m Throttled

One of the great things about FusionIO is that their drivers and software provide a crazy amount of information about the cards that are installed.

For example:

fio-status -a

This will return a whole lot of card information, like the currently installed cards, their serial numbers, power usage, power usage threshold, firmware, bytes read and written, and the estimated write life remaining on the card. Those are just the highlights, there’s a lot more there too.

It goes beyond this though, there are a whole lot of other metrics that can be pulled from the card. If you want a list

fio-status -l

will give you a complete list of things.

 

We’re actually interested in seeing whether or not we are getting throttled because of not giving enough power to the card. A quick little PS script (run as admin) can give us some nicely formatted information (use the version at the bottom of the page, this one just left in for historical purposes)

$PowerLimit = fio-status /dev/fct0 -F adapter.power_limit_watts
$ThrottleCount = fio-status /dev/fct0 -F iom.write_pwr_throttling_count
[datetime]$LastThrottle = ([DateTime]::Now).AddSeconds(-(fio-status /dev/fct0 -F iom.write_pwr_throttling_sec_since_last))

if ($ThrottleCount -gt 0)
{
write-output "Peformance on the FusionIO card has been throttled $ThrottleCount times, with the last event happening $LastThrottle."
write-output "The card currently has a max power draw of $PowerLimit. Consider increasing this to prevent throttling."
}
else
{
Write-Output "No power throttling detected."
}

 *UPDATE: There's a better version of this at the bottom of the page

What To Do If You Are Throttled

The first thing is to not make any dramatic changes, you don’t want to run into the potential of frying your motherboard or storage.

Check the card that you have installed and look at it’s max rating (you get can this from the FusionIO site). Once you have this, go through the cards that are installed in your server, and go through the server documentation to see what wattage gets delivered to the PCIe slots, and if that wattage is shared across cards.

If your servers PCIe slot can safely deliver extra power to the card then there is a Power Override software configuration option that can be set within FusionIO that will allow you to get the maximum performance from the card (no, I’m not going to give you that command, if you want to do this you need to do all the research, and be aware of the potential risks).

Is there much of a difference in performance? Could be more than 200MB a second. That’s not to be sniffed at.

 

Watching The Defaults

Much like with SQL Server installations, just blindly accepting default configurations can leave you in a place where everything is sub-optimal.  The same goes for Windows with things like Power Plans defaulting to a mid-level throttled state.

For everything you install and work with it is worth going through the default configurations and seeing if there is anything that you can do to make things run better.

 

Because Things Always Improve

Here’s a better PowerShell script that will iterate through the FusionIO devices on your system and return the results.

$fioInfo = fio-status.exe
$CardList = ($fioInfo | select-string "Attached") -replace("Attached", "")

foreach ($Card in $CardList)
{
$Card = $Card.Trim()
$cardLocation = "/dev/$Card"

$PowerLimit = (fio-status $cardLocation -F adapter.power_limit_watts)
$ThrottleCount = (fio-status $cardLocation -F iom.write_pwr_throttling_count)

if ($ThrottleCount -gt 0)
{
[datetime]$LastThrottle = ([DateTime]::Now).AddSeconds(-(fio-status $cardLocation -F iom.write_pwr_throttling_sec_since_last))
write-warning "$cardLocation - $ThrottleCount performance throttling events detected. Last event: $LastThrottle. Current max power draw: $PowerLimit Watts."
}
else
{
Write-Output "$cardLocation - No performance throttling detected. Current max power draw: $PowerLimit Watts."
}
}