A Lesson in Storage Costs

As the [former] Customer Service Manager at A Small Orange, comments, questions, and even complaints were frequently brought to my attention.

One common question is “why do you offer so little disk space compared to your competitors?” Or to phrase it another way, “why are your shared hosting plans so stingy?”

This is an understandable question. From the customer’s viewpoint, they can (as of 2010) buy a 1TB disk drive from Newegg.com anywhere from $59 to $89. “Drives are cheap!”, they cry. “Add more drives! I’ve got 3TB in my desktop PC, why can’t you do it on my server?”

Servers don’t quite operate this way.

For one thing, the hard drive in a server is expected to be reading and writing data almost constantly 24×7, instead of spinning up occasionally for downloads, torrents and gaming. That constant disk reading/writing will shorten the usable life of the drive. When a server’s hard drive dies, it’s not just one customer’s data that is lost…it can be several hundred customers’ data. So a server needs some failover, or redundancy. For a low-cost shared server, that might mean two drives in a mirrored (RAID-1) configuration so that one drive fails, the other one keeps going with a copy of the data until the first one can be replaced. A higher-end server might even have four or six drives, with some combination of mirroring or data protection (RAID 0+1 or RAID 5, for example) so that one or more drives can fail without the losing valuable customer data.

With this redundancy alone, we’ve gone from doubling the cost (two $90 drives for $180) to sextupling the cost (six drives at $534).

Now consider the performance of the drive. The cheapest $59 drive on Newegg would be be a “green” drive. It is a low power, low speed (5400RPM) drive, and has a very low cost per-Gigabyte. I have a couple of these drives in my home storage box and they run just fine. But I don’t access my data at home in the same way a web server will. There is an $89 drive on Newegg that is a little bit more powerful – mostly faster (7200RPM). It’s great for a PC desktop or home storage unit, and maybe even a low-traffic server.

But when you have constant usage, that drive needs to access server data fast and go to the next request because the slower the hard drive is, the fewer browser hits those websites can handle. The standard SATA drive has a 7200RPM rotation speed, and that’s pretty good for desktop usage and low-end servers. For higher performance, SCSI and the newer SAS drives are still the best. A 300GB SAS drive currently runs about $279 and they’re *fast*. They spin at 10,000-15,000 RPMs, and the SCSI protocol they use offers superior performance in server environments.

So now, instead of $180 for 1TB of storage (two SATA drives, mirrored), we now have higher performance drives giving us 300GB of mirrored storage for $558. That’s 300GB of fail-safe, high-performance server storage for $558 versus a single $89 1TB drive in your PC. Three times the cost, but twice the speed and performance.

For even better performance, using a hardware RAID controller instead of software RAID is important. This hardware is at least $200, so now you get 300GB for $758 or more.

Apples and Oranges indeed!

This should provide some perspective on the differences between consumer and business server storage. The next logical question is, how do other hosting companies provide so much more storage, like 50GB or more per customer?

First, many hosting companies use a control panel software (such as cPanel) on their servers. This software is used on a single server, with a limited number of hard drives (usually 2, 4, or 6) and turns it into a profit-machine. You have a set number of customers per server, and when you run out of space, you buy a new server with a limited number of drives and start filling it up with customers.

Using A Small Orange’s Medium hosting plan as an example, a single 1TB drive would allow up to 600+ customers to (1000GB divided by 1.5GB of space per customer). Of course, A Small Orange is not using single desktop hard drives, but mirrored high-performance drives. The drives actually are 146GB SAS drives, so about 97 customers could share that space if they all had the Medium plan.

The more customers a hosting company can stuff onto a single server, the more money you make. So many hosting companies use a technique called overselling. This means the hosting company will offer more space per customer (say, 10GB or 50GB or even unlimited storage) knowing that the most customers will never use it all. If every customer on that server filled up their 10GB or 50GB of space, the server drive space will fill up very quickly and cause problems. To prevent this from happening, there are clauses in the Terms and Conditions that define other limits, usually number of inodes. This prevents customers from filling up the disk with a large number of small files. Additionally, there are usually “fair usage” clauses with “unlimited storage” plans to allow the hosting company to suspend service if the customer is using more than a fair share of the overall disk amount.

With the price of Storage Area Networks and Network Attached Storage devices becoming more accessible to smaller operations, some hosting companies are moving away from fixed storage like physical disks. By implementing a high-performance network storage, a given server can increase it’s storage capacity on demand to meet the increasing storage demands of its customers. As this becomes more common, larger storage quotas (*without* overselling) will eventually become more commonplace.