Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Ema Nymton
Apr 26, 2008

the place where I come from
is a small town
Buglord
Problem description:

In 2012 I bought Western Digital Green WD30EZRX 3.0 TB HD. I was still using Windows XP at the time, which could not understand disks that big. I used the Acronis WD Align software Western Digital provides on their website, and was then able to use the whole drive by splitting it into two drives: a 2 TB ("chest1") and a 740 GB ("Jewelbox"). It makes windows think Jewelbox is another drive, not a partition.





I have 3 HDs, not 4. CrystalDiskInfo isn't fooled, but it only sees one partition on the drive:





This has worked OK. I upgraded to Windows Vista not long after and I'm still using it (it's affordable and it's not Win 8). I've been using an extra Win XP partition to do my backup and maintenance with Driveimage XML, but I want to start using Linux USBs to do that instead.
However, the 740 GB Jewelbox partition will not show up in Windows unless WD Align is installed, which in inconvenient, and it won't show up in Linux, which the WD Align tool doesn't work on. (Tried Knoppix, Puppy, and PartedMagic) The alignment process for Advanced Format drives according to WD Support: Windows = Super Easy! Not Windows = Super Hard.

Acronis WD Align in Windows Vista says everything is aligned right and offers no new options.



Attempted fixes:

In linux GParted can see chest1 and see that the WD Green is a single 3.0 TB drive, accesses the "chest1" partition fine, but only sees the Jewelbox partition as unallocated space. It can format it, but doing so will erase the whole drive:





I really want to fix this in a non-destructive way. I don't want to reformat the drive completely, although it would probably work. The HD in question is not my boot drive. It's not that I don't have backups; these are the backups. The WD 3.0 TB drive is where I am storing copies of all my physical DVDs, PlayStation discs, and backup disk images new and old. It took hours to make them all.

I did find that TestDisk in linux can actually locate the "lost" 740 GB Jewelbox partition, and can even see its files and directories! It says that the partition is "bad" as a primary partition, but changes to "OK" if it's not primary.





If I go ahead and use TestDisk to alter the characteristics of the 740 GB Jewelbox partition, would that harm the drive? Or should I use diskpart in the Windows command prompt to mark Jewelbox to "inactive"?
Would either of those even help without erasing everything?

System specs:
Western Digital Green WD30EZRX 3
AMD Athlon II, Quad-Core 2.9GHz
ASUS M4A77TD motherboard
6 GB SDRAM
OSes: Win Vista, Win XP, Linux various

Location:
USA

I have Googled and read the FAQ:
Yes, and googling has led to further confusion and less hope, since most of what I've found is people bitching about crappy support for Linux vis-à-vis Advanced Formart drives. Pages that do have solutions talk about blocksize and byte sectors and unix commands I'm afraid to use without knowing if I'll lose my data.


TL;DR
I'd like to able to use all of my Advanced Formart 3.0 TB HD in linux without reformatting it all over again, but support for this type of drive kinda sucks.

Adbot
ADBOT LOVES YOU

telcoM
Mar 21, 2009
Fallen Rib

Ema Nymton posted:

In 2012 I bought Western Digital Green WD30EZRX 3.0 TB HD. I was still using Windows XP at the time, which could not understand disks that big. I used the Acronis WD Align software Western Digital provides on their website, and was then able to use the whole drive by splitting it into two drives: a 2 TB ("chest1") and a 740 GB ("Jewelbox"). It makes windows think Jewelbox is another drive, not a partition.

The alignment process for Advanced Format drives according to WD Support: Windows = Super Easy! Not Windows = Super Hard.

I really want to fix this in a non-destructive way. I don't want to reformat the drive completely, although it would probably work. The HD in question is not my boot drive. It's not that I don't have backups; these are the backups.

Since your drive is already partitioned, alignment is not much of an issue: the partitions are where they are, unless you move your data elsewhere and start changing the partitioning.

The problem is that the Jewelbox partition is not listed in the standard partition table at the beginning of the disk. Since the WD Align makes it show up as a separate drive in Windows, it probably has a separate partition table somewhere close behind the end of the chest1 partition.

Linux has a powerful feature called "device-mapper". It takes (chunks of) disk device(s), optionally does something, then presents the result as another disk device. It is the power behind various Linux storage features, like disk encryption, LVM, RAID, snapshots and multipathing. You can make mappings on top of other device-mapper mappings, too: that's how in enterprise servers you can have encrypted snapshots of multipathed software-RAIDed LVM disk sets, if you need to. All these features are little more than a smart wrapper on top of some device-mapper modules.

But you know what?
Nobody said you cannot use the device-mapper directly, if it is useful to you...

Before getting down and dirty, you'll need to know two or three big numbers:
* the LBA block number where the Jewelbox partition starts on the disk.
* the Jewelbox partition size, in blocks.
* or the number of blocks available on the disk after the start of the Jewelbox partition; that works too.
(When I'm talking of blocks, I mean traditional 512-byte blocks unless otherwise noted.)

The Jewelbox partition size is easy: your Testdisk run probably already told it to you. It seems to be either 1 565 558 784 blocks, or 1 565 558 777 blocks. Divide by two to get (binary) kilobytes, then divide by 1024 and then again by 1024, and you'll get 746 (binary) gigabytes and change, which is the correct size for the Jewelbox partition.

(The difference between those two values is 7 blocks: it's possible there's some slack for alignment reasons. One 4096-byte Advanced Format block can hold 8 traditional 512-byte blocks after all...)

The specifications of your drive at the manufacturer's website list the total size of your disk as exactly 5 860 533 168 blocks.

The size of the chest1 partition is exactly 4 294 965 248 blocks according to Testdisk.
The computers usually count starting from zero, so if the chest1 partition starts from the earliest possible position, the exact beginning of the disk, it would be expected to occupy blocks #0 ... # 4 294 965 247.

If the Jewelbox partition is at the very end of the disk and we use the smaller of the two possible size values given by Testdisk, the largest possible block number for the beginning of the Jewelbox partition is:
5 860 533 168 - 1 565 558 777 = 4 294 974 391

So... that leaves a range of 4 294 965 248 ... 4 294 974 390. A range of 9142 blocks.
An old-school guy might go over that range with any tool that can hex-dump a disk block, trying to find the second partition table or the beginning of the Jewelbox partition manually.

Hell no. You have a machine that can do it for you.
Here's a little script.
code:
#!/bin/sh
RANGE_START=4294965248
RANGE_LEN=9142
WHOLEDISK_DEV=/dev/sdc
for i in $(seq 0 9142)
do
    BLOCK=$( expr $RANGE_START + $i )
    RESULT=$(dd if=$WHOLEDISK_DEV bs=512 skip=$BLOCK | file -)
    if [ "$RESULT" != "/dev/stdin: data" ]
    then
        echo "$BLOCK: $RESULT"
    fi
done
Save it as a text file (name it "find_jewels.sh" or whatever), mark it executable (chmod a+x find_jewels.sh) and run it as root.
It is going to read each block in the calculated range and use the "file" command to see whether it will look like a start of a partition.
I made it skip the most common non-result ("/dev/stdin: data", which means basically "no idea what this is"), but it will probably spew a lot of lines at you, so you'd better run it like this:
code:
$ sudo ./find_jewels.sh | less
Any line that has the form:
<block number>: /dev/stdin: x86 boot sector; <stuff...>
is a potential candidate: that means it looks like a partition table or a beginning of a partition at first glance.

Once you have one or a few candidate block numbers, you can try creating a device-mapper table for the Jewelbox partition.
In your case, it would be a simple one-line map of type "linear".
A device-mapper table builds a virtual disk device. Each line in the map identifies a chunk of disk space on some other disk device(s), which will be presented as a specific part of the virtual disk device.

We want to present the Jewelbox data starting at the beginning of the virtual disk device, and point to the actual starting location on /dev/sdc.
So the table will be:
code:
0 1565558784 linear /dev/sdc <real start block>
The values are: starting point in the virtual device, length of device (in blocks), map type ("linear"), the real device, and the block on the real device where the mapped partition really begins.

Replace the <real start block> with one of the candidate block numbers you found.
Save it as a text file (e.g. jewelbox.txt)
Then run:
code:
# dmsetup create jewelbox --table jewelbox.txt
If the syntax is correct, this will create a fake disk device named /dev/mapper/jewelbox.
If the block number was the actual start of the Jewelbox partition, you should now be able to mount the /dev/mapper/jewelbox device just like you mount /dev/sdc1. You'll just need to find some way to run the dmsetup & mount commands at boot time, and you've solved this problem.

But if it was the fake partition table for the Windows "Disk 3", it won't mount just like that... but "fdisk -l /dev/mapper/jewelbox" will show you a valid partition table that looks like Windows "Disk 3". It's time to use another tool: kpartx.

Kpartx may be packaged with device-mapper-multipath tools, but it is very useful even if you don't have big enterprise multipathed disks. If, for any reason, you have a disk device that includes a partition table, but no accompanying partition devices, kpartx can create them for you.

If necessary, find the package that contains the kpartx tool in your distribution, install it, and then after all the above steps, run:
code:
# kpartx -a /dev/mapper/jewelbox
Now you should have /dev/mapper/jewelboxp1, matching the first (and apparently only) partition of the "Disk 3", which you can mount.

If you're uncertain, you probably should mount the filesystem in read-only mode at first, and see if the contents are accessible.
You might also run "ntfsinfo -m /dev/mapper/jewelbox" or "ntfsinfo -m /dev/mapper/jewelboxp1", check the "Cluster Size" and "Volume Size in Clusters" values, multiply them together and divide by 512 to get blocks, and make sure that the result is smaller than or equal to the length value used in the device-mapper table. Otherwise the filesystem driver might try to access beyond the end of the mapped device, and device-mapper won't like that.

None of the commands and scripts I described will write anything on the disk on their own: they are only reading the disk and setting up a mapping that can be used to mount the Jewelbox partition. Only mounting the filesystem in writeable mode or otherwise actively writing to the /dev/mapper/jewelbox* device(s) might harm your data if the parameters are not correct.

If something seems off, undo the mappings and try again with a different candidate value:
"kpartx -d /dev/mapper/jewelbox" will clean up the /dev/mapper/jewelboxp* mappings, and "dmsetup clear jewelbox" or "dmsetup remove jewelbox" will undo the manually-created mapping (just the mapping, not your data!).

Ema Nymton
Apr 26, 2008

the place where I come from
is a small town
Buglord
:aaaaa:
Wow, telcoM, that is really impressive.

Alas, I ended up buying a used external 2 TB drive to back up everything just in case I messed it up, and I did. I had already tried using TestDisk to fix the partitions before I saw your post, caused the data to be lost, and just reformatted the partitions and started over. I probably wouldn't have had the talent to do it your way anyway :smith:

However, they are both accessible in Windows and Linux, as planned. I just did it the caveman brute way. :/

code:
 sdc      8:32   0   2.7T  0 disk 
--sdc1   8:33   0     2T  0 part /media/chest
--sdc2   8:34   0 746.5G  0 part /media/jewel2 
So now I can use it as my backup platform and use the USB Linux I carry with me at work without issue.

  • Locked thread