violate NTFS! Free the hard disk!

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > violate NTFS! Free the hard disk!

Notax: May 14, 2014

Does anyone have any idea how many Petabytes or Exabytes the entire computer network on Earth is missing because of an old Windows XP file system? I would like to look at how many systems run NTFS currently and evaluate the scope of the impact of the partition values being unable to fully address the hard drive's quota.

# ? Nov 18, 2016 03:07

Adbot: ADBOT LOVES YOU

# ? Apr 26, 2024 05:33

Judge Schnoopy: Nov 2, 2005; dont even TRY it, pal

Here's some rough math for you:

I read somewhere that NTFS loses 8% raw space of a drive.

Overall, between smartphones, tablets, laptops, and PCs, 11% are Windows. We'll go ahead and assume these are all XP or greater.
54% are Android, which we need to weight. Smartphones and tablets probably average in at 10 GB space, with 8 GB and 16 GB being the most popular options (there are a toooon more bottom of the barrel budget smartphones sold than the premium ones we see in commercials). Windows, with all those fancy terabyte drives, probably averages closer to 250 GB per device. So every Windows equals roughly 25 Android devices in regards to storage size. I'll go ahead and assume Mac and Linux average out to 250 GB per device too for simplicity sake.

So 52 million Androids, 300 million Apples, 300 million Windows, and 520 million 'Others' (https://en.wikipedia.org/wiki/Usage_share_of_operating_systems) gives us a raw data size of maybe 293,000,000,000 gigabytes, or 293 exabytes?

This USC article agreed with that number in 2011 so let's bump that up to 350 exabytes today. http://news.usc.edu/#!/article/29360/How-Much-Information-Is-There-in-the-World

Now according to my napkin calculations here Windows holds a 25% share of that, or 87.5 Exabytes.

We lost 8% of that to NTFS, or 7 exabytes of storage space. This is 2% of the worlds raw storage space.

I know 2% of a huge number is still a huge number but in the long term perspective of things, we're not missing out on much. We could rebuild that 2% on new storage devices in a couple months.

# ? Nov 18, 2016 03:34

BangersInMyKnickers: Nov 3, 2004; I have a thing for courageous dongles

Individual devices don't really matter in that regard and anyplace where it does matter (virtual, consolidated environments) have dedupe and compression above the filesystem that makes those inefficiencies small beans and irrelevant.

# ? Nov 18, 2016 16:04

Eletriarnation: Apr 6, 2005; People don't appreciate the substance of things...
objects in space.; Oven Wrangler

Just imagine how many bytes are wasted for parity in the 8b/10b encoding in SATA and Ethernet (and DP, and USB3, etc.)... the horror!

Eletriarnation fucked around with this message at 17:56 on Nov 18, 2016

# ? Nov 18, 2016 17:54

HalloKitty: Sep 30, 2005; Adjust the bass and let the Alpine blast

Notax posted:

Does anyone have any idea how many Petabytes or Exabytes the entire computer network on Earth is missing because of an old Windows XP file system? I would like to look at how many systems run NTFS currently and evaluate the scope of the impact of the partition values being unable to fully address the hard drive's quota.

How on earth is NTFS so wasteful? I have no idea really what you're referring to.

# ? Nov 18, 2016 18:02

craig588: Nov 19, 2005; by Nyc_Tattoo

I remember when switching to NTFS was the more effective use of space. If anyone knows about it and would find it entertaining to do the math on it, how bad would we be if we were still on FAT?

# ? Nov 19, 2016 04:54

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

HalloKitty posted:

How on earth is NTFS so wasteful? I have no idea really what you're referring to.

Well, one factor is that Microsoft requires the NTFS compression algorithm to meet a throughput standard on a minimum-system-requirement PC for any mounted NTFS filesystem, which means any sort of resource intensive compression is out and you get garbage compression ratios even if your 20-core Xeon scoffs at realtime LZMA on an SSD. Disk compression would be quote a bit less lovely if they aimed it at a throughput target for your PC and if you plug it into a 486 then it sucks to be you.

No idea what "unable to fully address the hard drive's quota" means though.

BangersInMyKnickers posted:

Individual devices don't really matter in that regard and anyplace where it does matter (virtual, consolidated environments) have dedupe and compression above the filesystem that makes those inefficiencies small beans and irrelevant.

I don't want to speak in absolutes but most performance-sensitive environments aren't even running NTFS (at least at a device level), they're running a SAN, probably on Linux or Solaris or something. If you want NTFS on top of that it gets allocated out of a ZFS pool or something.

Paul MaudDib fucked around with this message at 05:08 on Nov 19, 2016

# ? Nov 19, 2016 05:00

shame on an IGA: Apr 8, 2005

craig588 posted:

I remember when switching to NTFS was the more effective use of space. If anyone knows about it and would find it entertaining to do the math on it, how bad would we be if we were still on FAT?

How many flash cards and usb drives are in circulation right now because almost all of them are FAT32

# ? Nov 20, 2016 19:05

BangersInMyKnickers: Nov 3, 2004; I have a thing for courageous dongles

Paul MaudDib posted:

Well, one factor is that Microsoft requires the NTFS compression algorithm to meet a throughput standard on a minimum-system-requirement PC for any mounted NTFS filesystem, which means any sort of resource intensive compression is out and you get garbage compression ratios even if your 20-core Xeon scoffs at realtime LZMA on an SSD. Disk compression would be quote a bit less lovely if they aimed it at a throughput target for your PC and if you plug it into a 486 then it sucks to be you.

Good news from the world's most advanced operating system

code:

compact /?

Displays or alters the compression of files on NTFS partitions.

COMPACT [/C | /U] [/S[:dir]] [/A] [/I] [/F] [/Q] [/EXE[:algorithm]]
        [/CompactOs[:option] [/WinDir:dir]] [filename [...]]

  /EXE       Use compression optimized for executable files which are read
             frequently and not modified.  Supported algorithms are:
             XPRESS4K  (fastest) (default)
             XPRESS8K
             XPRESS16K
             LZX       (most compact)

# ? Nov 21, 2016 04:00

Farmer Crack-Ass: Jan 2, 2001; this is me posting irl

Notax posted:

Does anyone have any idea how many Petabytes or Exabytes the entire computer network on Earth is missing because of an old Windows XP file system? I would like to look at how many systems run NTFS currently and evaluate the scope of the impact of the partition values being unable to fully address the hard drive's quota.

First, I'm going to be a total pedant and point out that NTFS predates Windows XP, going back to the very first versions of Windows NT.

Second... are you referring to how a hard drive that's sold as a 2 TB drive shows up as 1.81 TB in Windows? Or something else?

# ? Nov 21, 2016 04:57

BangersInMyKnickers: Nov 3, 2004; I have a thing for courageous dongles

Listen I have a 600mb MFT on this 1.8tb volume and the .03% overhead is intolerable

# ? Nov 21, 2016 05:59

Adbot: ADBOT LOVES YOU

# ? Apr 26, 2024 05:33

HalloKitty: Sep 30, 2005; Adjust the bass and let the Alpine blast

Farmer Crack-rear end posted:

Second... are you referring to how a hard drive that's sold as a 2 TB drive shows up as 1.81 TB in Windows? Or something else?

If it's that, that would be hilarious, and has nothing to do with Windows or NTFS, but is simply a different way of calculating the same size.

# ? Nov 21, 2016 07:38

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > violate NTFS! Free the hard disk!