for a service pack to fix this you would have to either get a new install disk with the service pack included or use a utility like Nlite to slipstream the service pack into your install disk. Does anyone know if a simple fix would be to just download a linux iso, burn it to a disk, then boot the live cd and us gparted or qt parted to preformat the disk before you install. I have several of said live cd's already lying around so it would be a simple fix for me... if it works
Though this article says that drives with EARS in the drive model (eg. WD10EARS) will have 4k sectors, just a quick calculation from the drive specs on the WD website seems to indicate that these drives still have only 512B sectors.
Formatted Capacity: 1,000,204 MB = 1000204000000B
User Sectors Per Drive: 1,953,525,168
dividing, bytes per sector: 511.9995464527
Even the LBA printed on the label in the article says the same number of sectors: 1,953,525,168
The drives "pretend" to be 512 bytes per sector-drives, but are really 4 KiB sector-drives. Apparently they do this both to the BIOS and the dude reading the label - to avoid confusion I guess.
Honestly, are you that dense? Companies still make T-Bird parts for many times what they once sold for - try getting one from Ford for free, tell me how that turns out for you. Likewise, if you pay the $250 for MS support they'll often write little barely-tested patches to fix your particular bug. Microsoft, just like Ford, has every financial incentive to get you to upgrade, and will never do more than the bare minimum once you commit to not spending money with them ever again.
From my point of view, the big mistake was to use the 7 and 8 pin's jumper for XP workaround and sector shifting. There is WD align utility that solves the issue for old good XP. Now 10% of users will decide that two medicines will work even better than one, so they will both install the jumper, and then align the partitions with some aligment tool. Or, the users will get their systems shipped with XP and jumper installed, and later will upgrade to Windows7. What will happen? In all these cases users will get a 50% decreased performance and drive weared out earlier than necessary due to many read-change-write cycles.
These PINs should been used to just enable native 4KB sectors when pin removed! So anyone with Vista or later OS and NEW PC (with 4KB aware BIOS) can simply take out the jumper and enjoy knowing that now it works in native 4KB.
I would even put the default at 4KB. And put a big warning - XP and older OS users please install the jumper and run the utility after OS installation!
The only limitation would be that if upgrading OS the additional operation is needed - to remove jumper, and then install clean Windows7. Majority of users will be upgrading this way anyway. And going forward it would just be less and less users who need the "XP jumper".
What a mistake...
I am suprised who is taking a product decisions on WD. They release external disks with built-in SmartWare (many users called it CrapWare) that always pop-up as CD letter, and there is no way to have just a clean drive without space wasted for software of no use (most recent firmware allows to disable pop-up CD but users need to flash new firmware to the USB drives).
And now... they release good new 4KB technology and there is NO WAY (no jumper) to just pass 4KB on interface?
No chance to get whats needed (wanted), and obligatory pushing of whats not needed (not wanted). I am fan of WD drives because of their low power consuption, but obligatory SmartWare and no way to allow native 4KB sectors were two wrong product decisions.
I've bought this drive without knowledge of 4k technology and thus I've big problem because I'm using XP and W7 in my computer. I'd like to use this drive as single partition data drive, but will my data accesible from both OS? And then is it better to format it with W7 ( readable with xp w/out align software? ) or to format it with XP with the align software ( no trouble with W7 after? )? Or maybe I should change it for a normal 512bytes drive... If anyone can help I'd be very happy :)
Initialize the new drive and create both partitions with Windows 7, then you do not need to do anything extra. Windows7 will care about all alignments. Then you can install XP into some partition, if you cannot just use "Windows XP mode" inside Windows7.
I'd be willing to bet the people at http://hddguru.com">http://hddguru.com know how or will very soon what commands to use to tell if a drive is of the 4k variety.
The "problem" with Windows XP and earlier (and many other OSes) is that they create partitions which are not aligned on a 4KB boundary.
If you're installing XP on a new drive, the workaround is to partition the drive correctly before installing XP, and have XP install into the existing partition. You can use a Linux boot CD, but you might need to use command-line tools to ensure the partition is aligned, so that's not for novices.
So, what's the effect on performance when using a drive with 4KB physical sectors? If your partition is correctly aligned, there should be no performance decrease. The OS filesystem cluster size defaults to 4KB, so all reads and writes will normally be done in multiples of 4KB anyway. Since there is an effective areal density increase due to the larger physical sectors, you can expect 4KB-sector drives to have a higher data transfer rate than an equivalent 512-byte-sector drive.
So "it's all good", providing your partitions are aligned on a 4KB boundary.
If the partition is not correctly aligned, each filesystem read of n 4KB clusters requires the drive to read n+1 4KB physical sectors. The performance impact of that will be pretty small I think. Most filesystems have a read-ahead feature that reads in larger chunks anyway. The worst-case would be a 4KB random-read benchmark, and even there, no extra disk revolutions are needed.
Writes are the big problem. The impact will be decreased to some extent due to the drive's cache and the computer typically writing data in larger chunks. But read-modify write requires more revolutions of the disk. Each revolution takes 8.3ms for a 7200rpm drive. The worst-case scenario will be much slower than a 512-byte-sector drive.
Consider a 4KB random-write benchmark, i.e. one that writes 4KB of data to random places on the disk. (If your partition is not aligned, then none of the 4KB clusters are aligned.) For each 4KB write, the drive has to do this:
- Read two 4KB sectors.
- Wait while the disk makes one revolution (1/120th sec for a 7200rpm drive)
- Write back the two modified sectors
For every write in that case, an extra revolution of the disk is necessary. The actual worst-case scenario is even worse, but unlikely to be a problem in practice. Consider the case where the two sectors that the drive needs to read are on adjacent physical tracks. (The computer typically won't know where physical track boundaries are.) Then to write a 4KB sector, the drive needs to do this:
- Read the first sector (last sector of a track)
- Wait for a disk revolution
- Write the modified first sector
- Seek to the next track
- Read the second sector (first sector of the track)
- Wait for a disk revolution
- Write the modified second sector
So that requires two extra disk revolutions and a seek.
Solid-state drives have many of the same issues, along with a limited number of flash write cycles. Typically, the SSD flash page size is 4KB. If you use an SSD, you can improve drive performance and lifetime by ensuring your partitions are 4KB-aligned.
For what it's worth, drives with larger-than-512-byte sector sizes have been around for years. Magneto-optical (MO) drives first appeared in the late 1980s, and could use disks with 1024-byte sectors. (In that case though, the drive doesn't emulate a smaller sector size, so there is no problem.)
9.1GB MO drives, which first came out in late 2000, can use disks which have 4KB physical sectors, but the drive emulates 512- or 1024-byte sectors. Just like these new 4KB-sector hard drives.
DVD-RAM discs have 32KB physical sectors, but emulate 2048-byte logical sectors.
I think Steve Gibson better get off his arse one of these days and update Spinrite. I suppose it will probably still work on 4KB drives, but I suspect it will be VERY non-optimal. It needs better support for modern SATA interfaces anyway, so it's time.
"Along with the need for 4K-aware operating systems, drive imaging software needs to be 4K-aware. Otherwise such imaging software may inadvertently create misaligned partitions too. As such, Win 6.x is also affected by alignment issues when imaging software is used, as some (and perhaps all?) imaging products currently available will write misaligned partitions and/or clusters."
That is not true of file based imaging such as WIM.
When You clone to an empty drive, the disk cloning/imaging software must create the destination partition that MUST be 4K boundaries aligned so they MUST be 4K aware.
I can state it for sure, as I run into the problem!!! After cloning my 500GB to a 1TB WD drive I got a very slow system...
Problem solved using WD utility
they talk about using jumpers on the drive to provide various workarounds to the 512b address mapping.
what about a jumper to say "address w/ 512b" which is enabled by default and if you pull the jumper, the hdd will bypass that translation layer and allow direct 4kb addressing?
Yeah, when I initialized my 3x7200.11 raid5 array I had to use GPT.
Does anybody happen to know if Windows 7 (intel matrix raid) automatically align's the disk correctly? I have tried searching google but Windows 7 and Intel Matrix has me confused.
I am not sure if Windows 7 knows about the underlying disk structure of the matrix raid?
windows vista/2008 or higher are aware of alignment and norm correctly set the partitions correctly
if you run diskpart there is an option you can pick to show if it has been incorrectly aligned or not (not sure where thought or there is an tool that lists it)
If 4k will not be the standard till 2014(estimate), When will Sata 6.0 be implemented to improve the speed of Read-Modify-Write conditions.
I read the Seagate Barracuda runs Sata 6.0, BUT only for outer sectors.
Any word on a total solution for HD's?
hdparm tells me "Logical/Physical Sector size: 512 bytes" but this article tells me:
Unfortunately, at this point the utility also serves as the only way to identify an Advance Format drive without physically looking at it. Looking at the ATA spec, it looks like there’s a provision for asking a drive its native sector size (regardless of 512B emulation) but at this point there’s no such tool beyond the WD Align utility itself.
No, hdparm just passes along what the drive's controller reports to the kernel. The controller pretends the drive has a 512 byte sector size, even through physically this is not what happens when a write is done by the controller. The same thing already happens with SSDs -- on my Gentoo home server my SSD happily reports:
Logical Sector size: 512 bytes
Physical Sector size: 512 bytes
despite pages being far larger than that.
Any issues running mixed 4k and 512b drives in the same system? assume the OS (WinXP/Win7) will be able to sort it all out and transferring data from one to the other will occur properly?
It doesn't really make sense to me why they can't release 4K sector drives without the emulation and just write in big letters "For use with Windows Vista and higher" on it? Consumers now need to be careful about getting SATA vs. PATA drive, what is one more specification that they need to be aware of?
You're assuming users will have something resembling a clue. WD would get its return dept flooded by XP users reporting defective drives.
What they could theoretically do would be to have a jumper setting that changed if they exposed 512b emulation or 4k native sectors to the OS and defaulted it to 512b. This'd work universally out of the box, and would be a trivial fix for the clueful to switch to the optimal mode.
If the raw read is the only bottleneck in the system, you might theoretically get that, but in practice I doubt it's the only bottleneck. That is a good point though, there should be some performance improvement from this (particularly with very large reads.)
I have a couple 1.5 TB Caviar Green drives in a software RAID 5 array (Linux, mdadm, 64K stripe). Anyone know if I'd run into compatibility problems if I expanded the array to include my old drives plus a new 4K drive? I know I wouldn't be able to make use of the extra space, but the better error correction would be nice.
XP/2003 has a disk partition utility that allows setting offsets. I've used it in 2003 for some SAN environments to do partition alignments. It may not be as automated as the WD tool, but it does work on any drive.
We've been dealing with alignment issues in the server and SAN space for years, especially with virtualization at the vm level and storage level: wrapping data in vmdks on vmfs on wafl filesystems opens many opportunities for misaligned data and up to a 20% performance hit on saturated systems.
thankfully microsoft FINALLY recognized the issues with server 2008 and formats at a 1mb boundary automatically, which covers the majority of storage arrays and disks.
"Caviar Green... They all have 64MB of cache -
the first WD drives to come with that much cache..."
What about the 2TB Caviar Black and Raid Edition drives?
They have 64MB caches, and dual on-board cache-processors.
Is that all there is to their performance edge?
I didn't get in to it in the article, but there's also the benefit of getting rid of other per-sector overhead: sector gaps and sector sync marks. Between that and the ECC improvement, you get the total 7%-11% improvement.
Because it means they need 5% less process shrink for their next generation platters; probably 750GB. Computing less ECC code will let them get away with a marginally cheaper controller chip as well.
Implementing it in current models is mostly as a largescale testbed for the next generation although they could get an additional small performance gain from a 5% short stroke.
It would be great for any power user that understands the 5.xx limitations of these drive's to be able to purchase them regardless. I'm building a new linux file server for work and i'd love to drop in this new technology instead of knowing about and not being able to take full advantage of it for several years simply due to the fact that microsoft dropped the ball yet again.
I wonder why the "power user" would want these drives, since they come with a significant performance hit under certain conditions when writing - as mentioned in the article.
To write a 4KB block that spans two physical sectors, the sectors have to be read, data modified and then written. That means the platter has to spin an additional time - thats a pretty big 50% performance hit when it happens.
The benefits (reliability, capacity) are nice, but to me nothing to call home about. I backup daily and weekly. The last drive that failed me was 15 years ago. I'm content with how things are. But I'm not willing to take performance hits for these benefits. So, I hope they find a way around this issue before these drives are the only ones I can buy...
I'd imagine these "power users" would be the ones using an OS that knows what to do with 4k sectors, and would not have the performance problems XP would. And it would be nice if they had a jumper setting (enabled as shipped) to enable or disable the 512b emulation. Thus the only ones likely to disable the 512b emulation are the ones who know what they are doing.
This is my thought as well. When SATA2 was new and some first-gen SATA controllers were found to have problems with drives running in SATA2 mode, Seagate shipped their Barracuda 7200.8 (I think) with a jumper installed that disabled SATA2 by default. At the time there was little performance benefit unless you had support for NCQ the whole way up the stack and were multitasking heavily, so most users probably still to this day have that jumper installed whether they need it or not and are none the wiser. Those who knew what they're doing however could remove the jumper and gain a slightly faster burst transfer rate out of disk cache and the various protocol enhancements such as NCQ.
due to hdds not braking past 90MB/s the jumper on made there drives the most compatible drives around as i have had problems in the past with puling that jumper off on some motherboards (or none segate hdds) not working stable with Sata 2
same issue with Nvidia and Sata 2 on there apple systems wish SSDs had an option to set Sata 1 mode as the only SSD that you most likely going to always work is the corsair S128 as its Sata 1 only
So Mac OS X doesn't seemed to have problems with Advanced Format, but what about Boot Camp? Will XP Boot Camp still have issues while Vista and Windows 7 Boot Camp won't or does the Boot Camp partition utility automatically properly aligns things for even Windows XP.
I don't know if WD is answering followup questions, but could you ask if other virtual machines have this problem too? Specifically, I'm thinking of Windows 7's XP mode.
xp mode is an add-on to windows 7, as windows 7 will all ready be installed the XP mode install does not work at the disk level when it is installed (does not create partitions)
for vmware if its File based VMs it same as win7 XP mode its not working at disk level when partitions are been made inside the VMs(think VMware can be disk based thought but most use file based)
So do these drives still run in 512k emulation mode under Win7?
Or does Win7 recognize them as 4k drives thus partitioning and formating them accordingly?
They still run in 512B emulation. The emulation is done at the drive controller level; with these WD drives it can not be disabled. Win7 is only aware to the point that it knows not to start a partition at a misaligned logical sector.
Hmm, OK I misread the article. I thought that Windows Vista/7 properly addressed 4KB chunks and it was only XP that was stuck in 512B emu-hell. So the new drives operate at 4KB at the hardware level, but regardless of OS, they are 512B at the interface level? Do we need to wait then for the next batch of WD drives that operate and interface at 4KB in order for Vista/7 to take advantage of this?
I'm really looking forward to the 20% bump in useable storage space.
Yes. I'd also expect WD to ship some Black Edition drives that will be configurable between 512/4096 sectors on the interface, a long time before all shipped drive will default to 4096.
Yes, you need to wait for WD to ship drives that enable 4K sectors at the interface level. Due to the number of Win 5.x machines still in existence, this is likely a few years off in order to avoid the resulting grief for drive manufacturers. I suspect we're looking at 2014, but it could certainly be earlier.
I'm not sure exactly how 4K sectors will be exposed to the OS, but if the commands accessing the disks are different they could expose this without impacting old OS (by supporting both 512 and 4K access). Even if that's not possible, certainly WD could have made this switchable by jumper, so I can't see why WD would need several years to wait here.
In theory I guess 4K access should be faster (less command i/o) or at least have less cpu overhead, but maybe the gains just aren't significant at all hence WD didn't bother.
This is just a little bit misleading, (probably not intentionally though). The current batch of drives will store data in 4K chunks, but will allow the OS to use 512-byte aligned addresses. You will still get the effective capacity increase and improved error correction in the drive, this is transparent to the OS. From what I can tell the only problem is that the drive itself has to work harder, and that there are some newer file systems that can optimize things further if they had a 4K interface size.
Summary:
With the current batch of drives you get the space and ECC benefits of 4K sectors, but with some minor problems related to translation, which buys you compatibility with win XP.
With the next version of the tech (no idea when this will appear), the drives will remove the compatibility mode, and advanced filesystems will be able to optimize things further.
Couldn't the OS issue be fixed in a patch or service pack?
If all manufactures will be going to 4k then it seems like Microsoft needs to start working on a proper fix for this. Rather than leaving to the manfuactures, which might end up producing a multitude of different 'fixes'.
I've bought this drive without knowledge of 4k technology and thus I've big problem because I'm using XP and W7 in my computer. I'd like to use this drive as single partition data drive, but will my data accesible from both OS? And then is it better to format it with W7 ( readable with xp w/out align software? ) or to format it with XP with the align software ( no trouble with W7 after? )? Or maybe I should change it for a normal 512bytes drive... If anyone can help I'd be very happy :)
You'll be fine, but you MUST partition the drive from windows 7 or with another 4k-aware tool. Do NOT partition it with XP. Both of them will work absolutely fine after that, there's no other tweaks you have to make to get XP working, since it has 512b emulation.
Since the workarounds basically require the partition to shift across 512 bytes,
From the notes it looks like Windows 5.x will be able to read the drives without issue.
However the partition manager in the Windows installer on the original CD is broken and will not work. along with the partition manager in Computer Manager.
The windows installer didnt change much if at all from the blue screen wizard from NT 4 to Windows 2003 R2 (4.0 - 5.2)
Vista and 2003 use a different installer.
It is impossible or very unlikely that MS will issue new updated Windows Installation Disks that is the main problem/symptom.
If you create a Partition table in Vista / Windows 7 + you may be able to install from the 5.x CDs without trouble since the partitions already exist.
For the next few years the new 4 KB sector drives will emulate older drives and thus be able to be used on any system that could use 512 B sector drives, right back to DOS x.x. They just won't *perform* as well for unaligned writes as the old drives did save in cases where the partition alignment has been fiddled to make things line up more efficiently.
Note, by the way, that this partition alignment can be accomplished manually using older software: just adjust such that each primary partition starts on a physical 4 KB boundary (e.g., for the first partition on a disk start it on the 9th logical 'cylinder' of 63*255 512-byte sectors, since 8*63*255*512 is divisible by 4096). Logical partitions in the 'extended partition' need slightly different alignment to account for the fact that they don't include the first logical 'track' of 63 512-byte sectors, used by the 'extended partition boot record' in the same manner that the first logical 'track' on the disk is occupied by the 'master boot record'. Of course, mere partition alignment won't solve the performance problem if the file system residing in the partition doesn't observe neat 4 KB alignment internally.
(It also may be possible to change the size of the 'track' occupied by the MBR or EPBR to 64 512-byte sectors, which would be a cleaner solution but might confuse some software.)
Over the longer term, it's worth noting that Windows NT and its descendants have always been able to process data drives with sector sizes other than 512 bytes - they just can't boot from them. So save up a few boot drives for the time when no more 512-byte sector drives are available and keep using these systems as long as you care to.
it is fixed, in non ancient operating systems. Get windows 7...
oh, you mean you want microsoft to upgrade their old OS for free to make it compatible with new technology?
XP has reached its Extended Support phase. Microsoft will not be releasing anything besides security updates.
There may be other issues too in trying to make Win 5.x 4K-aware, but the fact of the matter is that they're not going to lift a finger while its in Extended Support anyhow.
I doubt it. But there are several (easy and viable) workarounds.
Companies that ship WHS systems may precreate the correct partitions and another option would be to create partition tables using a 4K aware partitioning utility, for example by booting into a Windows 7 Setup and creating the partitions there.
My concern is that WHS is kind of designed to last a long time. I don't know what a new build of WHS based upon Server08 is going to offer the consumer. I really envisioned my server staying as it is for a long time, only occasionally adding a drive to the pool as needed. I have a 14-bay case, plus the ability to add external racks, so I have a lot of potential growth. I would prefer not having a headache in 2011 when these drives are the standard. I also don't see the need to upgrade/replace my server until the hardware dies. It isn't like an OS that you use every day that can always use a revision.
Also. Has MS released any major core OS updates to WHS since the rest of windows 5.x went onto extended support? Patches to things that are WHS specific don't count. I'm asking about features that are/should be common with XP/2k3.
I wouldn't hold my breath. The HW companies have work around's available even if they're suboptimal; and WHS has a fairly small userbase to justify a major engineering effort. Especially since a new version of WHS based on server 08 is rumored to be coming next year and patching the current WHS would only draw attention to not patching XP/server 2k3.
Yes, there would be HUGE issues throughout the file system stack, and also bleeding over into page table management, memory management, DMA, drivers, and who knows what else. It's not a simple fix at all, and even if they weren't halting support for winXP, I doubt they'd be willing to make the necessary changes.
Linux went through this process a while back, and it was a very extensive set of changes even though Linux was more modular than windows to start with.
We're not talking about making XP 4K blocks aware, but simply let it create partitions aligned on 4K boundaries. This would be a trivial fix. But yes it's not a security issue so no luck.
btw I wonder what the performance is like if partition is unaligned. Noone tested?
I installed XP SP3 on an older master drive (40GB) Then from XP I choose to format the WD10EARS slave drive in logical sectors of 4K. Will it still be necessary to run the alignment tool on this Drive?
Does anyone know if WD 4K drives that were originally formatted on a Mac using OSX, will have any alignment problems when running XP on that Mac under Bootcamp, VM Fusion, Parallels, etc?
FYI, I remember back in the late 1970s when Wang transitioned from 256B sectors to 1K blocks when their hard drive storage cabinets (the size of a small desk!) increased to 80MB. No transition problems except for hernias.
What a nightmare these WD20EARS drives have given me with the 4k cluster overlap. The ReadyNas Pro speed is down to 1/5th the speed of 22MB/sec write. when tested with Bonnie++ and actual file copies.
A complete joke that WD states that it does not support Linux.
A complete Joke that Netgear has not tested these drives and got them working with their product as they are a Major HDD supplier.
So how will 4K affect RAID set ups? I setup several WD 20EARS drives in RAID 5, and of course Win7 only sees one' drive'. Has the RAID controller correctly aligned the drive?
The WD Align tool is no use as it sees the RAID array as a non-WD drive.
This is using the built in nVidia RAID on an Asus M3N78-EM. nVidia is silent on the issue, and the WD website doesn't discuss RAID in the context of AFT.
Correct me if I am wrong, but after reading this article on 4k sectors: http://seagate.com/docs/pdf/whitepaper/tp613_trans... I assume that it is possible to align Advanced Format drives (those ones, which are not provided with a special aligning software by HD manufacturer) for Windows XP usage with 3rd party 4k-aware partitioning utility: "The most critical aspect of a smooth and successful transition to 4K sectors used in Advanced Format is to promote the use of 4K-aware hard drive partitioning tools." "When using third-party software or utilities to create hard drive partitions, check with your vendor to make sure they are updated and confirmed to be 4K aware."
So to align properly for Win XP usage for example Samsung 2TB F4EG HD204UI one has to find a 3rd party software tool for partitioning, which is 4k-aware, and use that software to partition such disk instead of Windows XP Disk Manager.
Are there any free 4k-aware partitioning tools for Windows XP?
Also would re-aligning utilities provided by WD and Seagate work with other manufacturers HDs like Samsung?
From that article I learnt also that even the newer OSes like Windows 7 do not use 4k sectors directly, they still have to use 512 B sectors. Those newer systems are only capable of aligning Advanced Format hard drives properly during their partitioning, without the need to use 3rd party 4K-aware partitioning software, and that's all: "there are many aspects of modern computing systems that continue to assume that sectors are always 512 bytes. To transition the entire industry over to the new 4K standard and expect all of these legacy assumptions to suddenly change is simply not realistic. Over time, the implementation of native 4K sectors, where both host and hard drive exchange data in 4K blocks, will take place. Until then, hard drive manufacturers will implement the 4K sector transition in conjunction with a technique called 512-byte sector emulation."
"The sector size increase, described by Advanced Format, occurs at the hard drive media level. Host systems will continue to request and receive data from the hard drive in 512-byte sector sizes. However, the translation from 4096-byte sectors in the hard drive to the 512-byte sectors in the host will be managed in the hard drive. This process is called 512-byte emulation. It’s important that every drive partition start with an LBA offset that is aligned to the drive’s physical 4K sector. If partitions are un-aligned, then hard drive performance will be degraded." "How can partition misalignment conditions be managed? The first management step is to avoid misaligned conditions in the first place. This can be achieved by creating hard drive partitions with a 4K aware version of your operating system or through a hard drive imaging software product." "The second method to managing misaligned partitions is to use partition-alignment software to identify and fix misaligned partitions. This technique should be used during the hard drive integration process."
"It’s not practical to make host computer systems talk in 4K native language – at least for a while. There are just too many places the 512 byte assumption is embedded. So, while hard drives will transition to 4K sectors physical sectors on the media, they will still “look and talk” like 512 byte formats to host computers. They will actually emulate 512 byte communications to hosts. This works well as long as the logical 512 byte assumptions from the host computer are aligned with the 4K sectors on the hard drive." "It turns out that when a hard drive partition is created, the starting position can vary. A 4K drive format is set to work under the assumption that the first 512 byte sector (Logical Block Address = 0) will align perfectly with the first physical 4K sector". "Alignment 0 ... works well for hard drives & 512 byte emulation because they can neatly map eight 512 byte logical blocks into a single sector. Sometimes hard drive partitions get created so the logical to physical alignment is off...". "This is called Alignment 1 and it’s not so good for 4K drives when it comes to emulating 512 byte legacy sectors, especially when writing data. Essentially, this alignment can often cause a hard drive to manage a write with extra disc rotations, which slows things down. The results can be sort of dramatic as shown by these test results at HOTHARDWARE.com" "The Windows XP situation seems fairly well understood in the marketplace. What is much less understood is the situation with cloning and imaging software. System builders, integrators and IT organizations frequently rely on these tools to configure systems for sale or deployment in their organization. Even if you are using Windows Vista or Windows 7, if your hard drive was partitioned with one of these utilities, you’re likely to end up with an un-aligned partition with the potential for poor performance." http://consumer.media.seagate.com/2010/03/the-digi...
There seems to be much confusion and I feel as though I'm being thrown in. This seems to be, as far as I've been able to find, an x86 issue for XP users (not sure about Vista or 7). For those running Windows XP x64 Edition, this software layer interface to firmware level should not be an issue when it comes to alignment.
That said, however, I suspect that Windows XP x64 Edition still formats a new, raw unformatted hard drive into 512 bit sectors during install, thereby requiring either an external live CD or boot CD that can properly format the drive's partition sector mapping into the 4096 bit Advanced Format that exists at the firmware level. Otherwise, the alignment problem will appear as the partition addresses may attempt to locate a sector at a point partway into the larger section.
I have two new un-partitioned 1TB Black WD drives that I'll be using for data storage. Even though they don't indicate Advanced Formatting on them, If I partition them for 4k sectors and quick format them at 4096 allocation unit size will they work ok with XP PRO SP2?
"Farther down the road will be the exposure of 4K-sectors to the operating system itself. Linux and Win 6.x are set (and we believe that Mac OS X is too), the only limit right now is the desire to do a phased transition to make thing easier for legacy users. Since 4K-sector drives won’t work on Win 5.x at all, drive manufacturers can’t put them on the market so long as there’s a significant legacy base. 2014 – the year that extended support ends for WinXP – looks like a good year to finally complete the move to 4K sectors."
It's already happening... I just bought a 3TB Western Digital My Book Studio LX, Model WDBACH0030HAL-NESN. I chose this drive in because I wanted Firewire (it also has USB), and I did not want to go with the Seagate dockable drives due to reports (here) of overheating (the WD LX is in an well-ventilated aluminum case).
Many websites are listing this drive as compatible with Windows XP, so I ordered it from Amazon.
The box says "formatted for Mac OS X"... and "requires reformatting for Windows XP, Vista, or 7". In much smaller print, it says "Windows XP does not support drives with capacities greater than 2TB". I took that to mean that I can't create a partition over 2TB, which I expected anyways.
Upon connecting the drive to an XP PC, a 'virtual CD' was mounted with utilities. I ran the quick format utility, but it eventually fails saying something like "the sector size cannot be changed with your firmware version".
After fruitlessly searching for help, I called WD and they eventually told me this drive is not supported in XP, and there is no way to adjust the sector size or enable translation.
The primary purpose of this drive was to take an image of a Win2003 server using VMWare Converter from a laptop running XP. This turned out to be impossible, but I was able to put a firewire card in the server (it only had USB1.0) and make an image directly.
However, after that was done, I was planning to use this drive to backup my PCs, most of which are running XP. I have one machine running Win7 but it does not have Firewire; I may have to buy a firewire card. My PC at work is also XP, so there will be no way to use this drive to move data from home to work.
I am also disappointed that no major-brand 2-3tb drives seem to exist with 2 or more of the following: USB 3 eSATA Firewire
I know it's possible because some of the external case manufactures make cases with USB 2, eSATA, and Firewire 800 in on, but I've had too many bad experiences with third-party cases (poor ventilation, failed fans, failed power supplies, weak plug connections, ...)
Seagate comes close with their docks, but the base model is USB 3 only, and additional docks are $30-50 EACH, and they are bulky to carry around.
FYI, at least one MAC website lists this drive as being a NAS as well as having Firewire, but that is incorrect. From the pictures, however, it was impossible to tell as the Firewire 800 ports are indistinguishable from CAT5 in small pictures. Also, the USB port is a mini-USB so one has to always carry an (included) cable around with it if one wants to use USB.
Perhaps one day USB 3 will be the only connector we need for external storage, but I've had bad experience with high CPU usage on USB 2 and I don't see any evidence that USB 3 is better (still no DMA).
Advanced Format Drive Fix for Windows XP (Solution, workaround). After reading a lot of crap about Advanced Format drives having problems with Windows XP and other old OSes, I decided to post this easy fix. So if you want to use the Advanced Format Drive with other OSes than Vista or 7 do this one easy thing after you purchase the drive. Create all the partitions on the disk using Acronis Disk Director 11 (or later). Remember to tell Acronis Disk Director that you'll be using the drive for Windows 7, even though you'll use it for XP or other OS. Do not use any obsolete partitioning tools like Acronis Disk Director 10 or Windows XP disk management! The old partitioning tools will create the partitions misaligned! That's all you have to do! You can do a test and see if you have done everything right by running this utility: Dell Utility Advanced Format HDD Detection Tool. Just Google it. You'll see whether your partitions are Aligned or Misaligned. If you delete a partition using Windows XP and recreate it using Windows XP it will be misaligned again. So use Acronis Disk Director 11 to recreate it and everything will be fine again.
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
86 Comments
Back to Article
davidschorfhaar - Friday, February 12, 2010 - link
for a service pack to fix this you would have to either get a new install disk with the service pack included or use a utility like Nlite to slipstream the service pack into your install disk. Does anyone know if a simple fix would be to just download a linux iso, burn it to a disk, then boot the live cd and us gparted or qt parted to preformat the disk before you install. I have several of said live cd's already lying around so it would be a simple fix for me... if it worksiforum - Saturday, January 30, 2010 - link
Though this article says that drives with EARS in the drive model (eg. WD10EARS) will have 4k sectors, just a quick calculation from the drive specs on the WD website seems to indicate that these drives still have only 512B sectors.Formatted Capacity: 1,000,204 MB = 1000204000000B
User Sectors Per Drive: 1,953,525,168
dividing, bytes per sector: 511.9995464527
Even the LBA printed on the label in the article says the same number of sectors: 1,953,525,168
What am I missing?
runeks - Thursday, July 8, 2010 - link
The drives "pretend" to be 512 bytes per sector-drives, but are really 4 KiB sector-drives.Apparently they do this both to the BIOS and the dude reading the label - to avoid confusion I guess.
foxyshadis - Saturday, January 23, 2010 - link
Honestly, are you that dense? Companies still make T-Bird parts for many times what they once sold for - try getting one from Ford for free, tell me how that turns out for you. Likewise, if you pay the $250 for MS support they'll often write little barely-tested patches to fix your particular bug. Microsoft, just like Ford, has every financial incentive to get you to upgrade, and will never do more than the bare minimum once you commit to not spending money with them ever again.Algis - Monday, January 11, 2010 - link
I am, as many here, a technology entusiast.From my point of view, the big mistake was to use the 7 and 8 pin's jumper for XP workaround and sector shifting. There is WD align utility that solves the issue for old good XP. Now 10% of users will decide that two medicines will work even better than one, so they will both install the jumper, and then align the partitions with some aligment tool. Or, the users will get their systems shipped with XP and jumper installed, and later will upgrade to Windows7. What will happen? In all these cases users will get a 50% decreased performance and drive weared out earlier than necessary due to many read-change-write cycles.
These PINs should been used to just enable native 4KB sectors when pin removed! So anyone with Vista or later OS and NEW PC (with 4KB aware BIOS) can simply take out the jumper and enjoy knowing that now it works in native 4KB.
I would even put the default at 4KB. And put a big warning - XP and older OS users please install the jumper and run the utility after OS installation!
The only limitation would be that if upgrading OS the additional operation is needed - to remove jumper, and then install clean Windows7. Majority of users will be upgrading this way anyway. And going forward it would just be less and less users who need the "XP jumper".
What a mistake...
I am suprised who is taking a product decisions on WD. They release external disks with built-in SmartWare (many users called it CrapWare) that always pop-up as CD letter, and there is no way to have just a clean drive without space wasted for software of no use (most recent firmware allows to disable pop-up CD but users need to flash new firmware to the USB drives).
And now... they release good new 4KB technology and there is NO WAY (no jumper) to just pass 4KB on interface?
No chance to get whats needed (wanted), and obligatory pushing of whats not needed (not wanted). I am fan of WD drives because of their low power consuption, but obligatory SmartWare and no way to allow native 4KB sectors were two wrong product decisions.
tophe - Friday, January 15, 2010 - link
I've bought this drive without knowledge of 4k technology and thus I've big problem because I'm using XP and W7 in my computer. I'd like to use this drive as single partition data drive, but will my data accesible from both OS? And then is it better to format it with W7 ( readable with xp w/out align software? ) or to format it with XP with the align software ( no trouble with W7 after? )? Or maybe I should change it for a normal 512bytes drive... If anyone can help I'd be very happy :)Algis - Tuesday, March 9, 2010 - link
Initialize the new drive and create both partitions with Windows 7, then you do not need to do anything extra. Windows7 will care about all alignments. Then you can install XP into some partition, if you cannot just use "Windows XP mode" inside Windows7.ArchStanton1 - Friday, January 8, 2010 - link
Will these Advanced Format drives work with third-party drive encryption software?xpclient - Saturday, January 2, 2010 - link
This article (http://support.microsoft.com/default.aspx/kb/92333...">http://support.microsoft.com/default.aspx/kb/92333... : Windows Vista support for large-sector hard disk drives) lists about Windows support.toyotabedzrock - Wednesday, December 23, 2009 - link
I'd be willing to bet the people at http://hddguru.com">http://hddguru.com know how or will very soon what commands to use to tell if a drive is of the 4k variety.Donuts123 - Sunday, December 20, 2009 - link
The "problem" with Windows XP and earlier (and many other OSes) is that they create partitions which are not aligned on a 4KB boundary.If you're installing XP on a new drive, the workaround is to partition the drive correctly before installing XP, and have XP install into the existing partition. You can use a Linux boot CD, but you might need to use command-line tools to ensure the partition is aligned, so that's not for novices.
So, what's the effect on performance when using a drive with 4KB physical sectors? If your partition is correctly aligned, there should be no performance decrease. The OS filesystem cluster size defaults to 4KB, so all reads and writes will normally be done in multiples of 4KB anyway. Since there is an effective areal density increase due to the larger physical sectors, you can expect 4KB-sector drives to have a higher data transfer rate than an equivalent 512-byte-sector drive.
So "it's all good", providing your partitions are aligned on a 4KB boundary.
If the partition is not correctly aligned, each filesystem read of n 4KB clusters requires the drive to read n+1 4KB physical sectors. The performance impact of that will be pretty small I think. Most filesystems have a read-ahead feature that reads in larger chunks anyway. The worst-case would be a 4KB random-read benchmark, and even there, no extra disk revolutions are needed.
Writes are the big problem. The impact will be decreased to some extent due to the drive's cache and the computer typically writing data in larger chunks. But read-modify write requires more revolutions of the disk. Each revolution takes 8.3ms for a 7200rpm drive. The worst-case scenario will be much slower than a 512-byte-sector drive.
Consider a 4KB random-write benchmark, i.e. one that writes 4KB of data to random places on the disk. (If your partition is not aligned, then none of the 4KB clusters are aligned.) For each 4KB write, the drive has to do this:
- Read two 4KB sectors.
- Wait while the disk makes one revolution (1/120th sec for a 7200rpm drive)
- Write back the two modified sectors
For every write in that case, an extra revolution of the disk is necessary. The actual worst-case scenario is even worse, but unlikely to be a problem in practice. Consider the case where the two sectors that the drive needs to read are on adjacent physical tracks. (The computer typically won't know where physical track boundaries are.) Then to write a 4KB sector, the drive needs to do this:
- Read the first sector (last sector of a track)
- Wait for a disk revolution
- Write the modified first sector
- Seek to the next track
- Read the second sector (first sector of the track)
- Wait for a disk revolution
- Write the modified second sector
So that requires two extra disk revolutions and a seek.
Solid-state drives have many of the same issues, along with a limited number of flash write cycles. Typically, the SSD flash page size is 4KB. If you use an SSD, you can improve drive performance and lifetime by ensuring your partitions are 4KB-aligned.
For what it's worth, drives with larger-than-512-byte sector sizes have been around for years. Magneto-optical (MO) drives first appeared in the late 1980s, and could use disks with 1024-byte sectors. (In that case though, the drive doesn't emulate a smaller sector size, so there is no problem.)
9.1GB MO drives, which first came out in late 2000, can use disks which have 4KB physical sectors, but the drive emulates 512- or 1024-byte sectors. Just like these new 4KB-sector hard drives.
DVD-RAM discs have 32KB physical sectors, but emulate 2048-byte logical sectors.
nowayout99 - Saturday, December 19, 2009 - link
I can't imagine wanting one of these drives until after the transition. Who would want the performance hit?leexgx - Saturday, December 19, 2009 - link
windows 7 own disk backup i am quite sure is alignment aware as welljohnsonx - Saturday, December 19, 2009 - link
I think Steve Gibson better get off his arse one of these days and update Spinrite. I suppose it will probably still work on 4KB drives, but I suspect it will be VERY non-optimal. It needs better support for modern SATA interfaces anyway, so it's time.TechDicky - Friday, December 18, 2009 - link
"Along with the need for 4K-aware operating systems, drive imaging software needs to be 4K-aware. Otherwise such imaging software may inadvertently create misaligned partitions too. As such, Win 6.x is also affected by alignment issues when imaging software is used, as some (and perhaps all?) imaging products currently available will write misaligned partitions and/or clusters."That is not true of file based imaging such as WIM.
ppelleg - Sunday, February 21, 2010 - link
I think it's correst, instead!!!When You clone to an empty drive, the disk cloning/imaging software must create the destination partition that MUST be 4K boundaries aligned so they MUST be 4K aware.
I can state it for sure, as I run into the problem!!! After cloning my 500GB to a 1TB WD drive I got a very slow system...
Problem solved using WD utility
puffpio - Friday, December 18, 2009 - link
they talk about using jumpers on the drive to provide various workarounds to the 512b address mapping.what about a jumper to say "address w/ 512b" which is enabled by default and if you pull the jumper, the hdd will bypass that translation layer and allow direct 4kb addressing?
yuhong - Friday, December 18, 2009 - link
Don't forget the 2TB MBR barrier too, which will require either an increase in the sector size that is presented to software, or an move to GPT.hechacker1 - Friday, December 18, 2009 - link
Yeah, when I initialized my 3x7200.11 raid5 array I had to use GPT.Does anybody happen to know if Windows 7 (intel matrix raid) automatically align's the disk correctly? I have tried searching google but Windows 7 and Intel Matrix has me confused.
I am not sure if Windows 7 knows about the underlying disk structure of the matrix raid?
Thanks to anybody who knows.
leexgx - Saturday, December 19, 2009 - link
windows vista/2008 or higher are aware of alignment and norm correctly set the partitions correctlyif you run diskpart there is an option you can pick to show if it has been incorrectly aligned or not (not sure where thought or there is an tool that lists it)
ChuckR - Friday, December 18, 2009 - link
If 4k will not be the standard till 2014(estimate), When will Sata 6.0 be implemented to improve the speed of Read-Modify-Write conditions.I read the Seagate Barracuda runs Sata 6.0, BUT only for outer sectors.
Any word on a total solution for HD's?
pbhj - Friday, December 18, 2009 - link
hdparm tells me "Logical/Physical Sector size: 512 bytes" but this article tells me:Unfortunately, at this point the utility also serves as the only way to identify an Advance Format drive without physically looking at it. Looking at the ATA spec, it looks like there’s a provision for asking a drive its native sector size (regardless of 512B emulation) but at this point there’s no such tool beyond the WD Align utility itself.
So did hdparm just guess ....
GourdFreeMan - Friday, December 18, 2009 - link
No, hdparm just passes along what the drive's controller reports to the kernel. The controller pretends the drive has a 512 byte sector size, even through physically this is not what happens when a write is done by the controller. The same thing already happens with SSDs -- on my Gentoo home server my SSD happily reports:Logical Sector size: 512 bytes
Physical Sector size: 512 bytes
despite pages being far larger than that.
yacoub - Friday, December 18, 2009 - link
Any issues running mixed 4k and 512b drives in the same system? assume the OS (WinXP/Win7) will be able to sort it all out and transferring data from one to the other will occur properly?excalibur3 - Friday, December 18, 2009 - link
It doesn't really make sense to me why they can't release 4K sector drives without the emulation and just write in big letters "For use with Windows Vista and higher" on it? Consumers now need to be careful about getting SATA vs. PATA drive, what is one more specification that they need to be aware of?DanNeely - Friday, December 18, 2009 - link
You're assuming users will have something resembling a clue. WD would get its return dept flooded by XP users reporting defective drives.What they could theoretically do would be to have a jumper setting that changed if they exposed 512b emulation or 4k native sectors to the OS and defaulted it to 512b. This'd work universally out of the box, and would be a trivial fix for the clueful to switch to the optimal mode.
geofelt - Friday, December 18, 2009 - link
If the 4k sector is 10% more space efficient, would that not also also translate into 10% greater data transfer rate.azmodean - Friday, December 18, 2009 - link
If the raw read is the only bottleneck in the system, you might theoretically get that, but in practice I doubt it's the only bottleneck. That is a good point though, there should be some performance improvement from this (particularly with very large reads.)icrf - Friday, December 18, 2009 - link
I have a couple 1.5 TB Caviar Green drives in a software RAID 5 array (Linux, mdadm, 64K stripe). Anyone know if I'd run into compatibility problems if I expanded the array to include my old drives plus a new 4K drive? I know I wouldn't be able to make use of the extra space, but the better error correction would be nice.esandrs - Friday, December 18, 2009 - link
XP/2003 has a disk partition utility that allows setting offsets. I've used it in 2003 for some SAN environments to do partition alignments. It may not be as automated as the WD tool, but it does work on any drive.Iridium130m - Friday, December 18, 2009 - link
We've been dealing with alignment issues in the server and SAN space for years, especially with virtualization at the vm level and storage level: wrapping data in vmdks on vmfs on wafl filesystems opens many opportunities for misaligned data and up to a 20% performance hit on saturated systems.thankfully microsoft FINALLY recognized the issues with server 2008 and formats at a 1mb boundary automatically, which covers the majority of storage arrays and disks.
cjcoats - Friday, December 18, 2009 - link
"Caviar Green... They all have 64MB of cache -the first WD drives to come with that much cache..."
What about the 2TB Caviar Black and Raid Edition drives?
They have 64MB caches, and dual on-board cache-processors.
Is that all there is to their performance edge?
Ryan Smith - Friday, December 18, 2009 - link
That was supposed to be "the first WD Caviar Green drives...". Fixed.Byte - Friday, December 18, 2009 - link
Do the new WD drives give any space advantage? Also will there be any real performane advantage when they release the true higher level 4k clusters?magreen - Friday, December 18, 2009 - link
it sounds like you gain a whopping 220 bytes per 4Kb of storage space. That's 5%. Why does WD or anyone else care?
Ryan Smith - Saturday, December 19, 2009 - link
I didn't get in to it in the article, but there's also the benefit of getting rid of other per-sector overhead: sector gaps and sector sync marks. Between that and the ECC improvement, you get the total 7%-11% improvement.DanNeely - Friday, December 18, 2009 - link
Because it means they need 5% less process shrink for their next generation platters; probably 750GB. Computing less ECC code will let them get away with a marginally cheaper controller chip as well.Implementing it in current models is mostly as a largescale testbed for the next generation although they could get an additional small performance gain from a 5% short stroke.
Spivonious - Friday, December 18, 2009 - link
Because 5% of 1TB is 51GB? That's not exactly an insignificant amount.krazyderek - Friday, December 18, 2009 - link
It would be great for any power user that understands the 5.xx limitations of these drive's to be able to purchase them regardless. I'm building a new linux file server for work and i'd love to drop in this new technology instead of knowing about and not being able to take full advantage of it for several years simply due to the fact that microsoft dropped the ball yet again.Griswold - Friday, December 18, 2009 - link
I wonder why the "power user" would want these drives, since they come with a significant performance hit under certain conditions when writing - as mentioned in the article.To write a 4KB block that spans two physical sectors, the sectors have to be read, data modified and then written. That means the platter has to spin an additional time - thats a pretty big 50% performance hit when it happens.
The benefits (reliability, capacity) are nice, but to me nothing to call home about. I backup daily and weekly. The last drive that failed me was 15 years ago. I'm content with how things are. But I'm not willing to take performance hits for these benefits. So, I hope they find a way around this issue before these drives are the only ones I can buy...
strikeback03 - Friday, December 18, 2009 - link
I'd imagine these "power users" would be the ones using an OS that knows what to do with 4k sectors, and would not have the performance problems XP would. And it would be nice if they had a jumper setting (enabled as shipped) to enable or disable the 512b emulation. Thus the only ones likely to disable the 512b emulation are the ones who know what they are doing.wolrah - Friday, December 18, 2009 - link
This is my thought as well. When SATA2 was new and some first-gen SATA controllers were found to have problems with drives running in SATA2 mode, Seagate shipped their Barracuda 7200.8 (I think) with a jumper installed that disabled SATA2 by default. At the time there was little performance benefit unless you had support for NCQ the whole way up the stack and were multitasking heavily, so most users probably still to this day have that jumper installed whether they need it or not and are none the wiser. Those who knew what they're doing however could remove the jumper and gain a slightly faster burst transfer rate out of disk cache and the various protocol enhancements such as NCQ.leexgx - Saturday, December 19, 2009 - link
due to hdds not braking past 90MB/s the jumper on made there drives the most compatible drives around as i have had problems in the past with puling that jumper off on some motherboards (or none segate hdds) not working stable with Sata 2same issue with Nvidia and Sata 2 on there apple systems wish SSDs had an option to set Sata 1 mode as the only SSD that you most likely going to always work is the corsair S128 as its Sata 1 only
Mr Perfect - Friday, December 18, 2009 - link
Yes, that would be a great option. I can't be the only one who'd love to enable real 4KB sectors on all of my Windows 7 machines going forward.ltcommanderdata - Friday, December 18, 2009 - link
So Mac OS X doesn't seemed to have problems with Advanced Format, but what about Boot Camp? Will XP Boot Camp still have issues while Vista and Windows 7 Boot Camp won't or does the Boot Camp partition utility automatically properly aligns things for even Windows XP.Lukas - Sunday, December 20, 2009 - link
The bootcamp partitions aren't created by Windows XP, they're created by Mac OS X. Without having studied it, i'd bet that they will be aligned.Ryan Smith - Friday, December 18, 2009 - link
Good question; I don't have an answer to that.Mr Perfect - Friday, December 18, 2009 - link
I don't know if WD is answering followup questions, but could you ask if other virtual machines have this problem too? Specifically, I'm thinking of Windows 7's XP mode.leexgx - Saturday, December 19, 2009 - link
xp mode is an add-on to windows 7, as windows 7 will all ready be installed the XP mode install does not work at the disk level when it is installed (does not create partitions)for vmware if its File based VMs it same as win7 XP mode its not working at disk level when partitions are been made inside the VMs(think VMware can be disk based thought but most use file based)
pcfxer - Friday, December 18, 2009 - link
As of December 18, 2009, Bootcamp does NOT align the sectors any differently than the Microsoft installer would.flynace - Friday, December 18, 2009 - link
So do these drives still run in 512k emulation mode under Win7?Or does Win7 recognize them as 4k drives thus partitioning and formating them accordingly?
Ryan Smith - Friday, December 18, 2009 - link
They still run in 512B emulation. The emulation is done at the drive controller level; with these WD drives it can not be disabled. Win7 is only aware to the point that it knows not to start a partition at a misaligned logical sector.therealnickdanger - Friday, December 18, 2009 - link
Hmm, OK I misread the article. I thought that Windows Vista/7 properly addressed 4KB chunks and it was only XP that was stuck in 512B emu-hell. So the new drives operate at 4KB at the hardware level, but regardless of OS, they are 512B at the interface level? Do we need to wait then for the next batch of WD drives that operate and interface at 4KB in order for Vista/7 to take advantage of this?I'm really looking forward to the 20% bump in useable storage space.
Candide08 - Friday, December 18, 2009 - link
Wouldn't the 20% increase in storage still be realized - as long as the drives platters are formatted with the 4k sectors?The interface would be breaking the 4k transfers into 512 byte chunks for backwards compatibility, at a bit of a performance cost.
When the 4k interfaces are released then the full performance gains will be realized in addition to the storage efficiency increases, yes?
Lukas - Sunday, December 20, 2009 - link
Yes. I'd also expect WD to ship some Black Edition drives that will be configurable between 512/4096 sectors on the interface, a long time before all shipped drive will default to 4096.Ryan Smith - Friday, December 18, 2009 - link
Yes, you need to wait for WD to ship drives that enable 4K sectors at the interface level. Due to the number of Win 5.x machines still in existence, this is likely a few years off in order to avoid the resulting grief for drive manufacturers. I suspect we're looking at 2014, but it could certainly be earlier.mczak - Friday, December 18, 2009 - link
I'm not sure exactly how 4K sectors will be exposed to the OS, but if the commands accessing the disks are different they could expose this without impacting old OS (by supporting both 512 and 4K access). Even if that's not possible, certainly WD could have made this switchable by jumper, so I can't see why WD would need several years to wait here.In theory I guess 4K access should be faster (less command i/o) or at least have less cpu overhead, but maybe the gains just aren't significant at all hence WD didn't bother.
azmodean - Friday, December 18, 2009 - link
This is just a little bit misleading, (probably not intentionally though). The current batch of drives will store data in 4K chunks, but will allow the OS to use 512-byte aligned addresses. You will still get the effective capacity increase and improved error correction in the drive, this is transparent to the OS. From what I can tell the only problem is that the drive itself has to work harder, and that there are some newer file systems that can optimize things further if they had a 4K interface size.Summary:
With the current batch of drives you get the space and ECC benefits of 4K sectors, but with some minor problems related to translation, which buys you compatibility with win XP.
With the next version of the tech (no idea when this will appear), the drives will remove the compatibility mode, and advanced filesystems will be able to optimize things further.
therealnickdanger - Friday, December 18, 2009 - link
Let's hope so! Thanks for clarifying things for my simple brain. ;-)cjb110 - Friday, December 18, 2009 - link
Couldn't the OS issue be fixed in a patch or service pack?If all manufactures will be going to 4k then it seems like Microsoft needs to start working on a proper fix for this. Rather than leaving to the manfuactures, which might end up producing a multitude of different 'fixes'.
tophe - Friday, January 15, 2010 - link
I've bought this drive without knowledge of 4k technology and thus I've big problem because I'm using XP and W7 in my computer. I'd like to use this drive as single partition data drive, but will my data accesible from both OS? And then is it better to format it with W7 ( readable with xp w/out align software? ) or to format it with XP with the align software ( no trouble with W7 after? )? Or maybe I should change it for a normal 512bytes drive... If anyone can help I'd be very happy :)foxyshadis - Saturday, January 23, 2010 - link
You'll be fine, but you MUST partition the drive from windows 7 or with another 4k-aware tool. Do NOT partition it with XP. Both of them will work absolutely fine after that, there's no other tweaks you have to make to get XP working, since it has 512b emulation.carl0ski - Monday, December 28, 2009 - link
Since the workarounds basically require the partition to shift across 512 bytes,From the notes it looks like Windows 5.x will be able to read the drives without issue.
However the partition manager in the Windows installer on the original CD is broken and will not work. along with the partition manager in Computer Manager.
The windows installer didnt change much if at all from the blue screen wizard from NT 4 to Windows 2003 R2 (4.0 - 5.2)
Vista and 2003 use a different installer.
It is impossible or very unlikely that MS will issue new updated Windows Installation Disks that is the main problem/symptom.
If you create a Partition table in Vista / Windows 7 + you may be able to install from the 5.x CDs without trouble since the partitions already exist.
Bill Todd - Thursday, January 7, 2010 - link
For the next few years the new 4 KB sector drives will emulate older drives and thus be able to be used on any system that could use 512 B sector drives, right back to DOS x.x. They just won't *perform* as well for unaligned writes as the old drives did save in cases where the partition alignment has been fiddled to make things line up more efficiently.Note, by the way, that this partition alignment can be accomplished manually using older software: just adjust such that each primary partition starts on a physical 4 KB boundary (e.g., for the first partition on a disk start it on the 9th logical 'cylinder' of 63*255 512-byte sectors, since 8*63*255*512 is divisible by 4096). Logical partitions in the 'extended partition' need slightly different alignment to account for the fact that they don't include the first logical 'track' of 63 512-byte sectors, used by the 'extended partition boot record' in the same manner that the first logical 'track' on the disk is occupied by the 'master boot record'. Of course, mere partition alignment won't solve the performance problem if the file system residing in the partition doesn't observe neat 4 KB alignment internally.
(It also may be possible to change the size of the 'track' occupied by the MBR or EPBR to 64 512-byte sectors, which would be a cleaner solution but might confuse some software.)
Over the longer term, it's worth noting that Windows NT and its descendants have always been able to process data drives with sector sizes other than 512 bytes - they just can't boot from them. So save up a few boot drives for the time when no more 512-byte sector drives are available and keep using these systems as long as you care to.
taltamir - Friday, December 18, 2009 - link
it is fixed, in non ancient operating systems. Get windows 7...oh, you mean you want microsoft to upgrade their old OS for free to make it compatible with new technology?
Taft12 - Monday, December 21, 2009 - link
Yes, that is what I would like too. Service packs and patches have always been free.Calin - Sunday, December 20, 2009 - link
Yes, that would be great.Also, USB for Windows NT 4.0 would have been great at that time.
Ryan Smith - Friday, December 18, 2009 - link
XP has reached its Extended Support phase. Microsoft will not be releasing anything besides security updates.There may be other issues too in trying to make Win 5.x 4K-aware, but the fact of the matter is that they're not going to lift a finger while its in Extended Support anyhow.
mcnabney - Friday, December 18, 2009 - link
It appears the WHS is also impacted by this issue and is not in Extended Support. Should I be expecting a fix for my Home Server?Lukas - Sunday, December 20, 2009 - link
I doubt it. But there are several (easy and viable) workarounds.Companies that ship WHS systems may precreate the correct partitions and another option would be to create partition tables using a 4K aware partitioning utility, for example by booting into a Windows 7 Setup and creating the partitions there.
mcnabney - Monday, December 21, 2009 - link
My concern is that WHS is kind of designed to last a long time. I don't know what a new build of WHS based upon Server08 is going to offer the consumer. I really envisioned my server staying as it is for a long time, only occasionally adding a drive to the pool as needed. I have a 14-bay case, plus the ability to add external racks, so I have a lot of potential growth. I would prefer not having a headache in 2011 when these drives are the standard. I also don't see the need to upgrade/replace my server until the hardware dies. It isn't like an OS that you use every day that can always use a revision.DanNeely - Friday, December 18, 2009 - link
Also. Has MS released any major core OS updates to WHS since the rest of windows 5.x went onto extended support? Patches to things that are WHS specific don't count. I'm asking about features that are/should be common with XP/2k3.DanNeely - Friday, December 18, 2009 - link
I wouldn't hold my breath. The HW companies have work around's available even if they're suboptimal; and WHS has a fairly small userbase to justify a major engineering effort. Especially since a new version of WHS based on server 08 is rumored to be coming next year and patching the current WHS would only draw attention to not patching XP/server 2k3.azmodean - Friday, December 18, 2009 - link
Yes, there would be HUGE issues throughout the file system stack, and also bleeding over into page table management, memory management, DMA, drivers, and who knows what else. It's not a simple fix at all, and even if they weren't halting support for winXP, I doubt they'd be willing to make the necessary changes.Linux went through this process a while back, and it was a very extensive set of changes even though Linux was more modular than windows to start with.
mczak - Friday, December 18, 2009 - link
We're not talking about making XP 4K blocks aware, but simply let it create partitions aligned on 4K boundaries. This would be a trivial fix. But yes it's not a security issue so no luck.btw I wonder what the performance is like if partition is unaligned. Noone tested?
ppelleg - Sunday, February 21, 2010 - link
Yes... and it was terrible!!!After aligning the stuff using WD Align (the WD utility) the problem was solved and I get back performances...
evalfi - Monday, April 12, 2010 - link
I installed XP SP3 on an older master drive (40GB)Then from XP I choose to format the WD10EARS slave drive in logical sectors of 4K.
Will it still be necessary to run the alignment tool on this Drive?
Greetz & Thanks
ken1945 - Saturday, April 17, 2010 - link
Does anyone know if WD 4K drives that were originally formatted on a Mac using OSX, will have any alignment problems when running XP on that Mac under Bootcamp, VM Fusion, Parallels, etc?FYI, I remember back in the late 1970s when Wang transitioned from 256B sectors to 1K blocks when their hard drive storage cabinets (the size of a small desk!) increased to 80MB. No transition problems except for hernias.
MrCyberdude - Tuesday, May 11, 2010 - link
What a nightmare these WD20EARS drives have given me with the 4k cluster overlap.The ReadyNas Pro speed is down to 1/5th the speed of 22MB/sec write. when tested with Bonnie++ and actual file copies.
A complete joke that WD states that it does not support Linux.
A complete Joke that Netgear has not tested these drives and got them working with their product as they are a Major HDD supplier.
http://www.readynas.com/forum/viewtopic.php?f=24&a...
MrCyberdude
mfreesto - Sunday, September 12, 2010 - link
So how will 4K affect RAID set ups? I setup several WD 20EARS drives in RAID 5, and of course Win7 only sees one' drive'. Has the RAID controller correctly aligned the drive?The WD Align tool is no use as it sees the RAID array as a non-WD drive.
This is using the built in nVidia RAID on an Asus M3N78-EM. nVidia is silent on the issue, and the WD website doesn't discuss RAID in the context of AFT.
Anyone got any insights?
dragonf - Monday, November 8, 2010 - link
Correct me if I am wrong, but after reading this article on 4k sectors:http://seagate.com/docs/pdf/whitepaper/tp613_trans...
I assume that it is possible to align Advanced Format drives (those ones, which are not provided with a special aligning software by HD manufacturer) for Windows XP usage with 3rd party 4k-aware partitioning utility:
"The most critical aspect of a smooth and successful transition to 4K sectors used in Advanced Format is to promote the use of 4K-aware hard drive partitioning tools."
"When using third-party software or utilities to create hard drive partitions, check with your vendor to make sure they are updated and confirmed to be 4K aware."
So to align properly for Win XP usage for example Samsung 2TB F4EG HD204UI one has to find a 3rd party software tool for partitioning, which is 4k-aware, and use that software to partition such disk instead of Windows XP Disk Manager.
Are there any free 4k-aware partitioning tools for Windows XP?
Also would re-aligning utilities provided by WD and Seagate work with other manufacturers HDs like Samsung?
From that article I learnt also that even the newer OSes like Windows 7 do not use 4k sectors directly, they still have to use 512 B sectors. Those newer systems are only capable of aligning Advanced Format hard drives properly during their partitioning, without the need to use 3rd party 4K-aware partitioning software, and that's all:
"there are many aspects of modern computing systems that continue to assume that sectors are always 512 bytes. To transition the entire industry over to the new 4K standard and expect all of these legacy assumptions to suddenly change is simply not realistic. Over time, the implementation of native 4K sectors, where both host and hard drive exchange data in 4K blocks, will take place. Until then, hard drive manufacturers will implement the 4K sector transition in conjunction with a technique called 512-byte sector emulation."
"The sector size increase, described by Advanced Format, occurs at the hard drive media level. Host systems will continue to request and receive data from the hard drive in 512-byte sector sizes. However, the translation from 4096-byte sectors in the hard drive to the 512-byte sectors in the host will be managed in the hard drive. This process is called 512-byte emulation. It’s important that every drive partition start with an LBA offset that is aligned to the drive’s physical 4K sector. If partitions are un-aligned, then hard drive performance will be degraded."
"How can partition misalignment conditions be managed?
The first management step is to avoid misaligned conditions in the first place. This can be achieved by creating hard drive partitions with a 4K aware version of your operating system or through a hard drive imaging software product."
"The second method to managing misaligned partitions is to use partition-alignment software to identify and fix misaligned partitions. This technique should be used during the hard drive integration process."
"It’s not practical to make host computer systems talk in 4K native language – at least for a while. There are just too many places the 512 byte assumption is embedded. So, while hard drives will transition to 4K sectors physical sectors on the media, they will still “look and talk” like 512 byte formats to host computers. They will actually emulate 512 byte communications to hosts. This works well as long as the logical 512 byte assumptions from the host computer are aligned with the 4K sectors on the hard drive."
"It turns out that when a hard drive partition is created, the starting position can vary. A 4K drive format is set to work under the assumption that the first 512 byte sector (Logical Block Address = 0) will align perfectly with the first physical 4K sector".
"Alignment 0 ... works well for hard drives & 512 byte emulation because they can neatly map eight 512 byte logical blocks into a single sector. Sometimes hard drive partitions get created so the logical to physical alignment is off...".
"This is called Alignment 1 and it’s not so good for 4K drives when it comes to emulating 512 byte legacy sectors, especially when writing data. Essentially, this alignment can often cause a hard drive to manage a write with extra disc rotations, which slows things down. The results can be sort of dramatic as shown by these test results at HOTHARDWARE.com"
"The Windows XP situation seems fairly well understood in the marketplace. What is much less understood is the situation with cloning and imaging software. System builders, integrators and IT organizations frequently rely on these tools to configure systems for sale or deployment in their organization. Even if you are using Windows Vista or Windows 7, if your hard drive was partitioned with one of these utilities, you’re likely to end up with an un-aligned partition with the potential for poor performance."
http://consumer.media.seagate.com/2010/03/the-digi...
Video on 4K sectors:
http://usingwindowshomeserver.com/2010/07/27/4k-se...
Seagate manages aligment problems in the background without any interaction or knowledge of user.
BTW, a warning against 2TB hard drives:
http://www.clearfoundation.com/component/option,co...
imwithid - Sunday, November 14, 2010 - link
There seems to be much confusion and I feel as though I'm being thrown in. This seems to be, as far as I've been able to find, an x86 issue for XP users (not sure about Vista or 7). For those running Windows XP x64 Edition, this software layer interface to firmware level should not be an issue when it comes to alignment.That said, however, I suspect that Windows XP x64 Edition still formats a new, raw unformatted hard drive into 512 bit sectors during install, thereby requiring either an external live CD or boot CD that can properly format the drive's partition sector mapping into the 4096 bit Advanced Format that exists at the firmware level. Otherwise, the alignment problem will appear as the partition addresses may attempt to locate a sector at a point partway into the larger section.
Can anyone confirm this?
airesearch - Wednesday, January 5, 2011 - link
I have two new un-partitioned 1TB Black WD drives that I'll be using for data storage. Even though they don't indicate Advanced Formatting on them, If I partition them for 4k sectors and quick format them at 4096 allocation unit size will they work ok with XP PRO SP2?bobbozzo - Monday, March 14, 2011 - link
"Farther down the road will be the exposure of 4K-sectors to the operating system itself. Linux and Win 6.x are set (and we believe that Mac OS X is too), the only limit right now is the desire to do a phased transition to make thing easier for legacy users. Since 4K-sector drives won’t work on Win 5.x at all, drive manufacturers can’t put them on the market so long as there’s a significant legacy base. 2014 – the year that extended support ends for WinXP – looks like a good year to finally complete the move to 4K sectors."It's already happening...
I just bought a 3TB Western Digital My Book Studio LX, Model WDBACH0030HAL-NESN. I chose this drive in because I wanted Firewire (it also has USB), and I did not want to go with the Seagate dockable drives due to reports (here) of overheating (the WD LX is in an well-ventilated aluminum case).
Many websites are listing this drive as compatible with Windows XP, so I ordered it from Amazon.
The box says "formatted for Mac OS X"... and "requires reformatting for Windows XP, Vista, or 7".
In much smaller print, it says "Windows XP does not support drives with capacities greater than 2TB".
I took that to mean that I can't create a partition over 2TB, which I expected anyways.
Upon connecting the drive to an XP PC, a 'virtual CD' was mounted with utilities. I ran the quick format utility, but it eventually fails saying something like "the sector size cannot be changed with your firmware version".
After fruitlessly searching for help, I called WD and they eventually told me this drive is not supported in XP, and there is no way to adjust the sector size or enable translation.
The primary purpose of this drive was to take an image of a Win2003 server using VMWare Converter from a laptop running XP. This turned out to be impossible, but I was able to put a firewire card in the server (it only had USB1.0) and make an image directly.
However, after that was done, I was planning to use this drive to backup my PCs, most of which are running XP.
I have one machine running Win7 but it does not have Firewire; I may have to buy a firewire card.
My PC at work is also XP, so there will be no way to use this drive to move data from home to work.
I am also disappointed that no major-brand 2-3tb drives seem to exist with 2 or more of the following:
USB 3
eSATA
Firewire
I know it's possible because some of the external case manufactures make cases with USB 2, eSATA, and Firewire 800 in on, but I've had too many bad experiences with third-party cases (poor ventilation, failed fans, failed power supplies, weak plug connections, ...)
Seagate comes close with their docks, but the base model is USB 3 only, and additional docks are $30-50 EACH, and they are bulky to carry around.
FYI, at least one MAC website lists this drive as being a NAS as well as having Firewire, but that is incorrect. From the pictures, however, it was impossible to tell as the Firewire 800 ports are indistinguishable from CAT5 in small pictures.
Also, the USB port is a mini-USB so one has to always carry an (included) cable around with it if one wants to use USB.
Perhaps one day USB 3 will be the only connector we need for external storage, but I've had bad experience with high CPU usage on USB 2 and I don't see any evidence that USB 3 is better (still no DMA).
Thanks for the informative article, btw.
Bob
James5mith - Friday, May 27, 2011 - link
This article, and the 4K "conversion" has been happening for some time, and yet I have still been unable to find 4K native devices.I'm interested to see a rundown of what is and isn't possible with 4K drives, if there is any kind of throughput benefits and/or IOPS benefits, etc.
Do you know of anything coming within the next 6 months or so?
alpinemobile - Saturday, October 22, 2011 - link
Advanced Format Drive Fix for Windows XP (Solution, workaround).After reading a lot of crap about Advanced Format drives having problems with Windows XP and other old OSes, I decided to post this easy fix. So if you want to use the Advanced Format Drive with other OSes than Vista or 7 do this one easy thing after you purchase the drive.
Create all the partitions on the disk using Acronis Disk Director 11 (or later). Remember to tell Acronis Disk Director that you'll be using the drive for Windows 7, even though you'll use it for XP or other OS. Do not use any obsolete partitioning tools like Acronis Disk Director 10 or Windows XP disk management! The old partitioning tools will create the partitions misaligned!
That's all you have to do!
You can do a test and see if you have done everything right by running this utility: Dell Utility Advanced Format HDD Detection Tool. Just Google it.
You'll see whether your partitions are Aligned or Misaligned.
If you delete a partition using Windows XP and recreate it using Windows XP it will be misaligned again.
So use Acronis Disk Director 11 to recreate it and everything will be fine again.