File Server Builder's Guide
by Zach Throckmorton on September 4, 2011 3:30 PM ESTCPUs
Once again, the primary purpose of a file server is storage. It is not computation prowess nor producing high frame rates in games. All other components, including the CPU, should take a back seat to the hard drives, case, and power supply in the context of a home file server. File servers do not - repeat, do not - need the latest, greatest, more powerful processors to work well. In fact, file serving is not a particularly taxing task, especially not for a home file server that will likely never have to distribute data to more than a handful of clients simultaneously. Therefore, rather than using a powerful and power-hungry CPU, it's a better idea to use a less capable but also less power consuming chip.
From A(tom) to Z(acate)
Intel's Oak Trail (using Atom CPUs) and AMD's Brazos (using Zacate APUs) platforms are both up to the task of file serving. However, neither platform produces a particularly pleasant experience with Windows Home Server 2011. Both platforms take an agonizingly long time to install WHS2011, and neither will be capable of doing much more than simply serving - transcoding video on an Atom or Zacate WHS2011 system is painfully slow. That said, both Oak Trail and Brazos are sufficient to run WHS2011, especially if your file server will be performing only basic tasks like streaming MP3s and storing photos.
My preferred Atom home server motherboard/cpu combo is the ASUS AT5NM10T-I, a passively-cooled, Atom D525 (1.8GHz dual core with Hyper-Threading) solution that sports four SATA ports (rather than the two found on most Intel boards) and a PCIe x4 slot. The PCIe expansion slot is useful for adding a SATA controller card in case you want your file server to house more than four drives. Note it uses laptop SODIMMs rather than standard desktop DIMMs but considering how inexpensive DDR3 is currently, this does not affect the system's cost. One thing to keep in mind when selecting an Atom-based file server: go for the most recent models that are dual cores and have Hyper-Threading - the price premium is very modest and the performance increase is palpable.
ASRock's E350M1 is a more fully-featured Zacate motherboard that includes the E-350 APU (1.6GHz dual core), four SATA ports, an eSATA port (useful for backups), as well as VGA, DVI, and HDMI out ports. While multiple display outputs might not be an important consideration for a file server, more flexibility is always better. At just under $100 through Newegg currently, it is an exceptional value. Its expandability is limited to one PCIe x16 slot, which can also accommodate PCIe x4 and x1 cards - whereas PCIe 'up-plugging' can be hit or miss on different motherboards, I have had excellent success up-plugging on this particular model.
In comparing Atom with Zacate in the context of a file server, the regular laptop or desktop experience paints a useful picture. Atom is barely sufficient. Zacate is sufficient. Zacate's main strength is its integrated GPU, which is not particularly useful for a file server, but its CPU prowess is also substantially better than Atom's. While the Atom CPU officially draws less power on paper (with a TDP of 13W) than than the E-350 APU (at 18W), in practice, the two platforms consume very similar amounts of power both at idle and under loads typical of a file server (which do not tax the E-350's integrated GPU). Given that these platforms are priced similarly, use similar quantities of electricity, and the AMD platform's greater general flexibility, it is difficult to recommend the Atom-based solution given Zacate's substantial performance advantage.
The Sandy Bridge Pentiums
Anand recently reviewed the Sandy Bridge-based Pentiums, some of which have been available now for a few months. These CPUs are excellent home file server processors: they have enough muscle to smoothly run WHS2011 and produce a very pleasant computing experience, they use little power under load, and they are not expensive. The Intel Pentium G620 has become my go-to file server CPU since its release back in Q2. It is the least expensive Sandy Bridge desktop CPU at less than $80, and though its TDP is 65W on paper, in reality, it uses far less power under real world load. It uses even less power than its more powerful yet still juice-sipping Core i3-2100 cousin, and this combination of using not much electricity while being powerful makes it difficult to recommend any AMD CPU that is comparably priced. Unless you are on an extremely tight budget, a G620 makes more sense for a file server than, say, the AMD Athlon II X2 250. You can get an idea of how the G620 and 250 compare by looking at Bench - though since Bench doesn't include the 250, we're using the 255, which is ever so slightly faster than the 250. Pay particular attention to the power usage levels: at idle, the G620 system uses over 20W less than the 250 system, and uses less than 50W less under load.
But what about the lowly, dirt cheap Sempron 145? Its TDP is only 45W and though it's a single core CPU, it's still powerful enough for a file server, even one running WHS2011. Again, though, there's a difference between official TDPs and real world TDPs: my own testing shows that the Sempron 145 also idles nearly 20W higher than the G620. So the same conclusion immediately above also applies: unless you are on an extreme budget, you're better off with the G620 than the 145. These increases in CPU power consumption by the AMD CPUs compared to the Sandy Bridge Pentium translate directly into heat dissipated into your file server's case. Whether this is enough heat to make a difference depends on your case and cooling solutions - but in my experience, it's enough to push hard drive temperatures from the high 30s (Celsius) to to the mid 40s Celsius in especially small cases.
Though power consumption factors prominently in our recommendation of the Pentium G620 over the Athlon II X2 250, it's important to not lose sight of the forest for the trees: a 20W difference in power consumption for a file server CPU is the rough equivalent of leaving a smaller, lower wattage incandescent light bulb like a reading lamp on 24/7 in your home. Ultimately, the decision is simple: is a $25 or more premium for the G620 worth saving 20W+ on your electrical bill over the long haul?
The lilliputian Intel Pentium G620 heatsink and fan is a good indication of the chip's heat output
Motherboards
Whether you use a mini-ITX, micro-ATX, or full-size ATX motherboard will largely be dictated by the size of the case you decide to house your home file server in.
Mini-ITX
Mini-ITX boards sacrifice expandability for small size. Few ITX motherboards have more than four SATA ports, limiting them to use in file servers that will hold maximally 12TB (the largest commercially available hard drives are 3TB). However, most ITX motherboards have at least one expansion slot, which can hold a SATA controller card. There are many LGA 1155 motherboards that are compatible with the Intel Pentium G620, and most have very similar feature sets. However, in a mini-ITX case, board layout becomes critically important. My favorite ITX 1155 motherboard is Giada's MI-H61-01 specifically because its four SATA ports are clustered on the lower right aspect of the board (when it's mounted), right by the front panel connectors and 20 pin ATX power port. Because the four pin CPU power connector is located on the upper left corner, cable management is a breeze and facilitates excellent airflow - everything goes to either the lower right or upper left corners, allowing cables to be run along the top or bottom of the mini-ITX case. The Giada MI-H67-01 has a nearly identical layout and is sometimes less expensive than the H61 board. Though Giada is a newcomer to the North American market and do not have the reputation of older brands like ASUS, for what's it worth, I have used many of these boards in both file server and regular ITX desktop builds and have been completely satisfied by their products. Remember, if you want to build a mini-ITX file server that will have six hard drives, you will need to buy a PCIe SATA controller card with two SATA ports like the SYBA SD-SA2PEX-2IR or Rosewill RC-211.
Micro-ATX
Micro-ATX LGA 1155 boards can sport up to seven total SATA ports (4 SATA II and 3 SATA III), but most come with four or six total SATA ports. As with mini-ITX file server boards, layout is important when stuffing many hard drives into a micro-ATX case. The Biostar TH67B places all six of its SATA ports at the very bottom right-most corner of the board. Unfortunately there are no micro-ATX 1155 options that push the 20+4 pin power connector to either the very top or bottom of the board, but at least this Biostar board has its four pin power connector at the very top.
Full ATX
Cable management is rarely as difficult in a full-size ATX case as it can be in micro-ATX and mini-ITX cases, so board layout is perhaps less important for a full ATX file server motherboard, but it is still a consideration. Ten hard drives can become very messy! ATX 1155 boards max out at 10 ports, however, ten SATA port boards are typically $200 or more, whereas eight SATA port boards like the ASUS P8P67 can be found for as little as $125. Thus, if you absolutely need ten HDDs in your file server, it makes more sense to spend $125 on the board and $25 on a two port SATA controller card than $200 on a ten SATA port motherboard. I like this ASUS board for multiple HDD systems because its SATA ports are mounted perpendicularly to the board facing forward and at about the same height as one PCIe x1 and one PCI slot, so managing the SATA cables facilitates better airflow than if they were coming off of different heights on the board.
RAM
File servers do not need high performance, low latency, high frequency RAM. FreeBSD, FreeNAS, and Ubuntu all run well with 2GB of RAM under loads typical of a home file server, but they do run palpably smoother with 4GB. WHS2011 runs much more smoothly with 4GB. All of the file server OS's run even better with 8GB RAM. As RAM prices continue to fall, 8GB kits have been available for less than $30 (after rebate) regularly. Because RAM prices are so dynamic lately, rather than recommending a specific product, we'll recommend that you shop around! You should be able to find 2GB, 4GB, or 8GB for $5/GB without a rebate or less than $5/GB after rebate.
Now that we've covered CPUs, motherboards, and RAM, the next page discusses cases and PSUs solutions.
152 Comments
View All Comments
noxipoo - Sunday, September 4, 2011 - link
i'm looking for alternatives to drobo or the more expensive NAS devices so some raid card recommendations along with all the things that one needs would of been nice.bamsegrill - Sunday, September 4, 2011 - link
Yeah, some raidcard-recommendations would be nice.Rick83 - Sunday, September 4, 2011 - link
MY RAID card recommendation is a mainboard with as many SATA ports as possible, and screw the RAID card.For anything but high end database servers, it's a waste of money.
With desktop boards offering 10 to 12 SATA ports these days, you're unlikely to need additional hardware, if you chose the right board.
Otherwise, it's probably wisest to go with whatever chipset is best supported by your OS.
PCTC2 - Sunday, September 4, 2011 - link
But there's the fact that software RAID (which is what you're getting on your main board) is utterly inferior to those with dedicated RAID cards. And software RAIDs are extremely fickle when it comes to 5400 RPM desktop drives. Drives will drop out and will force you to rebuild... over 90 hours for 4 1.5TB drives. (I'm talking about Intel Storage Matrix on Windows/ mdadm on Linux).You could run FreeNAS/FreeBSD and use RAID-Z2. I've been running three systems for around 5 months now. One running Intel Storage Matrix on Windows, one running RAID-Z2 on FreeBSD, and one running on a CentOS box on a LSI2008-based controller. I have to say the hardware has been the most reliable, with the RAID-Z2 in a close second. As for the Intel softRAID, I've had to rebuild it twice in the 5 months (and don't say it's the drives because these drives used to be in the RAID-Z2 and they were fine. I guess Intel is a little more tight when it comes to drop-out time-outs).
A good RAID card with an older LSI1068E for RAID 5 is super cheap now. If you want a newer controller, you can still get one with an LSI2008 pretty cheap as well. If you want anything other than a giant RAID 0 stripe (such as RAID 5/6/10), then you should definitely go for a dedicated card or run BSD.
Rick83 - Sunday, September 4, 2011 - link
I've been using 5400 rpm disks and mdadm on linux for quite a while (6 years now?) and never had a problem, while having severly sufficient performance.If disks drop, that's the kernel saying that the device is gone, and could be jut a bad controller.
I've been on three boards and never had that kind of issue.
Windows is a bit more annoying.
Also, your rebuild time is over the top.
I've resynced much faster (2 hours for 400GB - so 10x faster than what you achieved. While also resyncing another array. Sounds like you may have a serious issue somewhere)
The compatibility advantage of software RAID outweighs any performance gain, unless you really need those extra 10 percent, or run extreme arrays (RAID-6 over 8 disks, and I might consider going with a dedicated card)
I think it might be the Intel/Windows combination that is hurting you - you might want to try the native windows RAID over the Intel Matrix stuff. Using that is the worst of both worlds, as you have the vendor lock in off a dedicated card and the slightly lower performance of a software solution.
Of course, you also mentioned mdadm, but I've never had a problem with that, with a bunch of different disks and controllers and boards.
I guess in two to three years, when I upgrade my machine again, I will have to look at a SATA controller card, or maybe sooner, should one of my IDE disks fail without me being able to replace it.
I think you may just have been unlucky with your issues, and can't agree with your assessment :-/
Flunk - Sunday, September 4, 2011 - link
I agree, I've used the Windows soft raid feature a lot and it trumps even hardware raid for ease of use because if your raid controller dies you can just put your drives in any windows system and get your data off. You don't need to find another identical controller. Performance is similar to matrix raid, good enough for a file server.vol7ron - Monday, September 5, 2011 - link
Wouldn't a RAID card be limited to the PCI bus anyhow? I would suspect you'd want the full speed that the SATA ports are capable ofvol7ron - Monday, September 5, 2011 - link
Even with 5400RPM drives, if you have a lot of data you're copying/transfering, you could probably saturate the full bandwidth, right?Rick83 - Monday, September 5, 2011 - link
PCI is wide enough to support gigabit ethernet, so if you don't have too many devices on the bus, you'll be fine until you have to build a RAID array.With PCI-X and PCIe these limitations are no longer of relevance.
Jaybus - Monday, September 5, 2011 - link
There are plenty of PCI-E x4 and x8 RAID cards.