bit-tech.net

Microsoft waves goodbye to FAT with exFAT deals

Microsoft waves goodbye to FAT with exFAT deals

Future peripherals could be Windows-only, if Microsoft succeeds in pushing its exFAT file system as a replacement for the ageing FAT format.

Microsoft has begun the long process of phasing out the FAT (File Allocation Table) filesystem and replacing it with exFAT, signing agreements with five hardware vendors to licence the format for use in peripheral devices.

First introduced in 1977 as an eight-bit storage format for Microsoft's BASIC language, the format received its last serious upgrade in 1996 with the launch of FAT32 as part of Windows 95 OEM Service Release 2 (OSR2.) Designed to extend the existing FAT16 format, FAT32 expanded the possible size of a partition on a hard drive from 2GB to 16TB but retained a file size limit that meant no single file could be larger than 4GB. While extensions to the format, including the open FAT+ standard, attempted to extend this to 256GB per file, few gained commercial traction in mainstream operating systems.

A lack of journalling and other data protection systems has led to a migration away from FAT and towards the New Technology File System (NTFS) first introduced in Windows NT 3.1 in 1993. Featuring a maximum file size of 16TB - expanded to 256TB in the NTFS implementation used by Windows 8 - the format is significantly more robust and adds POSIX-inspired features like hard links missing from FAT.

While NTFS has all but replaced FAT on the Windows desktop - while due to be replaced itself by the ReFS format used by Windows 8 - the lowly FAT format still rules the roost on removable devices where the 4GB file size limit doesn't matter. Its open nature, coupled with its wide compatibility with most mainstream operating systems - with full read and write support in every version of Windows since 95 OSR2, Apple's OS X and Linux - made it an obvious choice.

Now, however, Microsoft is looking to migrate people across to exFAT. First developed for Windows Embedded CE 6.0 in 2006, exFAT is non-backwards compatible but fixes many of the limitations of FAT32 including adding support for files larger than 4GB, improving storage efficiency for files smaller than 64KB, and lowering seek times in directories containing thousands of smaller files.

So far, exFAT's closed-source and proprietary nature, and its use of software patents to prevent third-party support being added without a licence, has limited its appeal. That appears to be changing, however, with Microsoft announcing deals with Sharp, Sigma, NextoDi, Black Magic and Atmos Global to add exFAT support to future products. Sharp will be adding the format to its future Android tablets, Sigma and NextoDi to its professional cameras and image storage products, and Black Magic and Atmos Global to their video recording systems.

These peripherals will be limited to use with selected operating systems thanks to a lack of exFAT support in third party products. Should they prove successful with consumers, the days of cross-platform devices could be drawing to a close and while Apple has licensed the format for OS X, open-source operating systems like Linux are likely to be left out in the cold.

47 Comments

Discuss in the forums Reply
theshadow2001 8th November 2012, 10:55 Quote
Another move to make Microsoft a more walled in Apple like eco system. I don't like it.
rollo 8th November 2012, 11:12 Quote
Fat is 35 years old and is a broken piece of crap that nobody uses on there pc anymore anyway.

NTFS has largely replaced all FAT systems in business, Windows 7 does not even offer the option of formating the drive as FAT last i checked.

The only software this will effect is linux which is likely to be left out in the cold.
mm vr 8th November 2012, 11:22 Quote
exFAT cannot be supported under Android, at least not as a kernel mode driver. It would have to be released under the GPL.

OSR2 is short for "OEM Service Release", by the way.
steveo_mcg 8th November 2012, 11:25 Quote
Quote:
Originally Posted by rollo
Fat is 35 years old and is a broken piece of crap that nobody uses on there pc anymore anyway.

NTFS has largely replaced all FAT systems in business, Windows 7 does not even offer the option of formating the drive as FAT last i checked.

The only software this will effect is linux which is likely to be left out in the cold.

And any hardware manufacturers who do not wish to licence the new fat from MS...
Griffter 8th November 2012, 11:31 Quote
sooo linux which is likely to be left out in the cold i guess. ;P
Gareth Halfacree 8th November 2012, 11:50 Quote
Quote:
Originally Posted by rollo
Windows 7 does not even offer the option of formating the drive as FAT last i checked.
Actually, you *can* format drives as FAT32 under modern Windows - but only *external* drives. The option doesn't appear by default for internal drives, nor for drives above 32GB. The functionality still exists in Windows 8, 'cos otherwise you wouldn't be able to format memory cards ready for use with digital cameras and the like - all consumer-grade version of which still use FAT.
Quote:
Originally Posted by mm vr
exFAT cannot be supported under Android, at least not as a kernel mode driver. It would have to be released under the GPL.
Sure it can be supported under Android: a project has been working on a kernel module for exFAT support since 2009, while a FUSE module also exists. 'Course, they're not suitable for inclusion in the Linux kernel itself - but the GPL says nothing about developers writing their own code for use on their own devices, while Microsoft will happily grant an exFAT licence to anyone willing to pay.
Quote:
Originally Posted by mm vr
OSR2 is short for "OEM Service Release", by the way.
Huh, I always thought it was "Operating System Release 2." You learn something new every day... I'll update the article - ta!
steveo_mcg 8th November 2012, 11:50 Quote
Quote:
Originally Posted by Griffter
sooo linux which is likely to be left out in the cold i guess. ;P

I realise that was tongue in cheek but I'll continue any way :D.

Computers are not the only places you'll find FAT, anything with internal storage will most likely be running a FAT file system. So your camera, your television, web streaming blu-ray the list goes on.

A free, (relatively) reliable and most importantly ubiquitous file system makes life much simpler for all us geeks not just Linux users .
isaac12345 8th November 2012, 12:40 Quote
The fact that this filesystem is proprietary makes this decision seem like more of a money making thing rather than 'fixing' an old filesystem. If they truly wanted to 'fix' it they would make it open like FAT.
NikoBellic 8th November 2012, 12:41 Quote
I really hope to see a new file format in my next smartphone in early 2013, because I am one of those few people that can actually go over 4GB in a single file (I have a 12GB 1080p movie) and at the moment it is annoying when you have to find ways in which you can divide those files yet still use them on your devices

PS: I know some people would think it is not really needed having a movie displayed in 1080p on a mobile device, but when I shrunk it down to my device resolution of 800x480, you could see that the film looked horrid, yet, when I divided the file and watched it in 1080p, it looked amazing... just a shame about the pauses you get when watching the film and half way through it goes to the next file...

And yes I do know that you can format the sdcard to ntfs on some android devices & it'll work with a few mods, but I can't risk bricking my phone.
azazel1024 8th November 2012, 12:57 Quote
Is it really that hard to get along and just come up with one standard file format across all OSs? Or at least one common file format for removable media?

I like me some Windows, but seriously, it is annoying as hell when I have my external HDD along and want to transfer a whack of files to my cousin at the holidays....and since he has a Mac Book Pro...no go, as it refuses too with the NTFS formating on the 640GB external drive. So then I am stuck slapping the images on a thumb drive and transfering them in 2 or 3 steps because the only way they can talk to each other is through FAT32 (at least storage to storage).

Sigh. I really hope exFAT doesn't become standardized and leave people out in the cold. I am not a huge Linux fan, but it deffinitely has its place. It'll be a sad/bad day if, for example, my only option on my future digital camera is to use it with Windows, or a Mac or I can get a special open source digital camera that sucks and will work with Linux.
isaac12345 8th November 2012, 13:20 Quote
Azazel1024, that's the scenario I envisaged too while reading the article. It'll be a mighty pain in the ass. And I'm sure its not hard for such massive companies to come up with one standard format. It's just a matter of will, financial and political.
Shirty 8th November 2012, 13:22 Quote
What has Microsoft got against fat people anyway? Steve Ballmer is quite fat.
dyzophoria 8th November 2012, 13:22 Quote
Quote:
Originally Posted by azazel1024
Is it really that hard to get along and just come up with one standard file format across all OSs? Or at least one common file format for removable media?

I like me some Windows, but seriously, it is annoying as hell when I have my external HDD along and want to transfer a whack of files to my cousin at the holidays....and since he has a Mac Book Pro...no go, as it refuses too with the NTFS formating on the 640GB external drive. So then I am stuck slapping the images on a thumb drive and transfering them in 2 or 3 steps because the only way they can talk to each other is through FAT32 (at least storage to storage).

Sigh. I really hope exFAT doesn't become standardized and leave people out in the cold. I am not a huge Linux fan, but it deffinitely has its place. It'll be a sad/bad day if, for example, my only option on my future digital camera is to use it with Windows, or a Mac or I can get a special open source digital camera that sucks and will work with Linux.

rather than leave exFAT on the cold I wish it doesn't become a closed standard so that it can be adapted by everyone faster, well unless everyone licenses exFAT. lol, i don't really want to stick to fat32 anymore honestly
Guinevere 8th November 2012, 13:35 Quote
Quote:
Originally Posted by NikoBellic
when I shrunk it down to my device resolution of 800x480, you could see that the film looked horrid, yet, when I divided the file and watched it in 1080p, it looked amazing

You had the resources of the entire internet and a desktop PC and you couldn't down-sample a 1080p video to 480p without it looking much worse than the same resizing performed during playback? i think if you'd have persevered you'd have been able to come up with something equally as good if not better.
rollo 8th November 2012, 14:57 Quote
NTFS is the future for internal storage

Wether exfat takes off for external storage time will tell.
proxess 8th November 2012, 14:59 Quote
Linux will be fine. It's the same as NTFS was. FUSE will solve all the issues.
Alecto 8th November 2012, 15:15 Quote
FAT is not missing hard links - you can have any entry pointing to a cluster already in use and pointed to by another entry. This constitutes a hard link. Might not be exactly standard (and FS checking software will report it as error) but it can be done.
PCBuilderSven 8th November 2012, 16:13 Quote
Quote:
open-source operating systems like Linux are likely to be left out in the cold.

Really? How long will it take reverse engineer it and add compatibility? Isn't NTFS also proprietary, and that works perfectly.
mi1ez 8th November 2012, 16:37 Quote
I actually came across an XP machine only today with 4x FAT32 partitions on one disk! It took me a good 10 minutes to figure out why I couldn't paste the 4.2 GB PST file to it despite some 40GB of free space!
XXAOSICXX 8th November 2012, 18:27 Quote
I really don't understand all the hate.

Microsoft makes something, wants to licence it out to make some "money". They're not forcing you to use it and even if they were, what difference would it make? As has already been mentioned, NTFS is proprietary we're (almost) all content with that.

Why do people act like Microsoft (et al) are anything other than businesses out to make a buck?
theshadow2001 8th November 2012, 19:50 Quote
The hate comes from the fact that instead of creating a foundation where these companies(linux,apple,microsoft, openBSD et al) get together and try and create a universal standard for things like this, propriety standards are used to try and lock people in. Ultimately it just creates frustration when the end users end up with compatibility problems. Companies like Microsoft make billions in profit, instead of doing something that benefits all end users they just try and eke more out. If there were more players in the O/S scene than the big three moves like this would be a bloody nightmare.
Icy EyeG 8th November 2012, 19:56 Quote
Quote:
Originally Posted by theshadow2001
Snip

THIS.
Adnoctum 8th November 2012, 20:49 Quote
This is a non-issue, and we are all carrying on about nothing. exFAT will be no more an issue on Linux than FAT is now. There isn't much urgency to make a solution at the moment because no one is using exFAT yet. When people do, proper support will occur. If you NEED it now (why?) there are solutions (such FUSE or Tuxera) now.

And as to being a closed-source Microsoft file system, well so is NTFS and I bet most Linux users would format their large capacity USB drives with NTFS, if only because of the wide compatibility with systems.
Is it ideal? No, an open solution would be best, but in the absence of a favoured implementation or a influential pushing supporter, exFAT may end up being it.
The only issue you might have is the Microsoft Tax that is added to devices that support exFAT, and that has already been paid via licencing so...
ArthurB 9th November 2012, 05:53 Quote
Quote:
Originally Posted by azazel1024
I have my external HDD along and want to transfer a whack of files to my cousin at the holidays....and since he has a Mac Book Pro...no go, as it refuses too with the NTFS formatting on the 640GB external drive.
Mac's have had NTFS *read* support for years so that should have worked. What I have sometimes found is that if you don't eject the HDD cleanly on Windows, OS X will often refuse to mount it. Running "chkdsk x: /f" from a Command Prompt usually fixes this issue.
Quote:
Originally Posted by azazel1024
So then I am stuck slapping the images on a thumb drive and transferring them in 2 or 3 steps because the only way they can talk to each other is through FAT32 (at least storage to storage).
If your HDD was formatted as exFAT you wouldn't have had this problem since OS X has supported exFAT since 10.6.5.
Bindibadgi 9th November 2012, 05:54 Quote
WinFS please!

Oh, wait...

Saivert 9th November 2012, 09:34 Quote
why don't everybody just use ext4 ?
fodder 9th November 2012, 10:42 Quote
Oh great, back to the 1980's days of Macs and PCs not being able to share storage mediums. Mind you, most files are transferred online now, not discs.
Anfield 9th November 2012, 11:36 Quote
Btrfs should be the future, but no, everyone keeps brewing up their own standards.
Star*Dagger 10th November 2012, 03:10 Quote
Everyone in the EU reading this should explain to their MEPs that this is BAD for both the PC platform in general, and the EU specifically.

I am sure that the EU will smash this one way or another, no worries.

Enjoy the show.
Woodstock 10th November 2012, 06:37 Quote
Microsoft cant really force exFat on you, but a gadget manufacturer using exFat on say an SD card could. Phones and cameras (video esp) being the prime suspects.
Woodstock 10th November 2012, 06:37 Quote
Also not sure how, they are being a bussiness, makes it all ok
Panos 10th November 2012, 12:53 Quote
Quote:
Originally Posted by Griffter
sooo linux which is likely to be left out in the cold i guess. ;P

When was the last time someone used FAT on Linux?
I use ext2 since the 90s when Linux came out, ext3 later on, and the last 5 years ext4.

There are far superior techs out there
theshadow2001 10th November 2012, 12:57 Quote
Quote:
Originally Posted by Panos
When was the last time someone used FAT on Linux?

Whenever someone plugged in a usb flash drive I suppose.
ShinyAli 10th November 2012, 14:46 Quote
There are some free FAT/FAT32 formatting programs available if for some reason it can't be done with an OS ;)
BLC 13th November 2012, 08:20 Quote
Quote:
Originally Posted by Panos
When was the last time someone used FAT on Linux?
I use ext2 since the 90s when Linux came out, ext3 later on, and the last 5 years ext4.

There are far superior techs out there

Me, last night when I had to format a USB stick as FAT32 so that I could transfer Puppy Linux Live to it in order to rescue my PC. Oh, and on the boot partition for my Raspberry Pi.

FAT32 does suck, but it's also pretty much universal when it comes to creating bootable USB drives. I've had a lot of trouble getting NTFS or EXT2/3/4 formatted USB drives to boot successfully - when I'm not using pre-built .IMG files and dd/Win32DiskImager, that is - but FAT32 usually works perfectly.
lp rob1 15th November 2012, 17:50 Quote
I tried ext4 on a USB drive once - since then I have never used it again for that purpose. The problem isn't the filesystem - that bit is fine, if not better than FAT32. You hit problems with permissions - ext4 has filesystem permissions, so if you create a file on the ext4 drive as UID 1000 (the default Ubuntu first user UID) then try to go and read it where your UID is 1001 (a second user perhaps?) the filesystem will stop you. Across multiple computers, which is usually where USB drives are used, this becomes an even greater problem. If there was a way to disable filesystem permissions for ext4 then I would be using that... Of course, if there were a filesystem, completely opensource like the ext family, that specifically lacked permissions support (along with other things that are irrelevant on a USB drive, like 'aggressive' journalling) then USB drives would finally have a place in the Linux ecosystem.
Gareth Halfacree 16th November 2012, 08:23 Quote
Quote:
Originally Posted by lp rob1
You hit problems with permissions - ext4 has filesystem permissions, so if you create a file on the ext4 drive as UID 1000 (the default Ubuntu first user UID) then try to go and read it where your UID is 1001 (a second user perhaps?) the filesystem will stop you. Across multiple computers, which is usually where USB drives are used, this becomes an even greater problem.
That's not a problem with ext4, that's a problem with not understanding how file permissions work. Each file or directory has three permissions: User, Group and Everyone. If you set a file to 700, the user will have full read, write and execute permission - but nobody else will. Set it to 770, and the user plus his or her group (typically 'users') will have full read, write and execute permission - but nobody else. Set it to 777, and the user, his or her group, and absolutely everyone capable of reading an ext4-formatted volume will have full read, write and execute permission.

You may notice that the latter option is functionally identical to your "way to disable filesystem permissions for ext4."

You can set these permissions on files manually ("chmod -R 777 *" for example), or you can set a 'create mask' at the point of mounting. Set the mask right, and you've basically got an ext4 volume with FAT32-like levels of security - i.e. none at all.

As for journalling, you can disable journalling altogether, or change how aggressive it is. Check the manpage for tune2fs for details.
lp rob1 16th November 2012, 17:59 Quote
Ah, but the problem with chmod'ing is that you have to do that every time you write new files in, unless you have the fancy mounting options set. And for those fancy mounting options - it is not guaranteed that can be done to each computer that I may need to work with - on some of the computers I do not even have permissions to mount anything, and FUSE (or whatever the mounting backend is) does it automatically, which unfortunately doesn't have the option of changing those mount options.

Because of the inherent multiple computer problem, setting things that only affect the local computer (like mount options) are out of the question. Changing the permissions of my files every time I create a new file is a temporary solution, but the annoyance caused by forgetting to do that one time then being stuck without access to files for, say, a day, is frustrating to say the least. Until permissions can be turned off on the filesystem itself I can only stay productive while using FAT32, or exFAT once Linux FUSE modules come out.
Gareth Halfacree 16th November 2012, 18:54 Quote
Quote:
Originally Posted by lp rob1
Ah, but the problem with chmod'ing is that you have to do that every time you write new files in, unless you have the fancy mounting options set.
Not true: you can set the default creation mask just by typing "umask" and the octal mask for the permissions you'd like to set.

Here, try it: grab a flash drive, format it as ext2/3/4 (whichever.) Create a file, check the file permissions - they're not what you're after, right? Type "umask 000" and create a second file. Check the permissions of *that* file.

The second file, you'll notice, has its permissions set such that anybody has read and write access to it. Every file you create from then on - in that session, naturally - will also have the same permissions, unless you manually override them. No need to set mount options, no need to change the permissions manually.
Quote:
Originally Posted by lp rob1
[...] the annoyance caused by forgetting to do that one time then being stuck without access to files for, say, a day, is frustrating to say the least.
Why would you be stuck without access to the files for a second, let alone a day? Just take ownership of the files: "chown yourusername *" Doesn't matter what the UID was, it'll reset it to your current UID. File permissions won't let you chown? "sudo chown yourusername *" There's no need to wait until you can go back to the computer that created the file and change the permissions!
lp rob1 17th November 2012, 00:06 Quote
Quote:
Originally Posted by Gareth Halfacree
Not true: you can set the default creation mask just by typing "umask" and the octal mask for the permissions you'd like to set.

The problem in this is just as you say - it only applies to the session. I want a permanent solution. Although saying that, umask may be useful in other circumstances...
Quote:
Why would you be stuck without access to the files for a second, let alone a day? Just take ownership of the files: "chown yourusername *" Doesn't matter what the UID was, it'll reset it to your current UID. File permissions won't let you chown? "sudo chown yourusername *" There's no need to wait until you can go back to the computer that created the file and change the permissions!

I am not trusted enough to have sudoers, or root access to most of the computers I use. Therefore this method will not work reliably.

As an example: on my home computer I am UID 1000 (or 1001, but it makes no difference). If I use umask or chmod every time, then I will have no problems. Unfortunately this is a command line annoyance - one of the things that GoodBytes always presses as a major problem of Linux. When I then take this USB drive to a different computer (for example the school computers), where I have a UID of 20106 AFAIK, then I will not have any access to my files that I did not prepare properly. Now on most of the computers that I use regularly I know the administrator password and so can use sudo to fix the problem, but there are a great many computers where this will not work. And besides, I may be using a completely different computer somewhere where I am a guest. Surely I won't have sudo or root access there...
Don't get me wrong - I would gladly use an open source filesystem for my USB drive, but until the permissions thing is somehow sorted, I cannot use one with any sort of efficiency or practicality.
Gareth Halfacree 17th November 2012, 10:53 Quote
Quote:
Originally Posted by lp rob1
The problem in this is just as you say - it only applies to the session. I want a permanent solution. Although saying that, umask may be useful in other circumstances...
Put your chosen umask in your profile, and it will apply to your own computer permanently. See here for details.
Quote:
Originally Posted by lp rob1
As an example: on my home computer I am UID 1000 (or 1001, but it makes no difference). If I use umask or chmod every time, then I will have no problems.
See above: if it's your home computer, you *do* have access to the mount options for your removable drive and can set the create mask there, right? You can also pass mount options through FUSE - something I neglected to mention earlier. Alternatively, you can set umask through your .bashrc or the system-wide profile to have it apply automatically to all files.
Quote:
Originally Posted by lp rob1
Unfortunately this is a command line annoyance - one of the things that GoodBytes always presses as a major problem of Linux.
Not so: see above for three different methods of setting it up to apply to all sessions with no command-line fiddling (and no risk of forgetting to do it.)
Quote:
Originally Posted by lp rob1
When I then take this USB drive to a different computer (for example the school computers), where I have a UID of 20106 AFAIK, then I will not have any access to my files that I did not prepare properly.
See above: if you set your home computer up right - the one computer you *do* have root access on, and therefore can configure accordingly - you will never have a single problem using ext2/3/4 on a removable drive.
Quote:
Originally Posted by lp rob1
And besides, I may be using a completely different computer somewhere where I am a guest. Surely I won't have sudo or root access there...
You misunderstand me: the only computer you ever need root access on is *your* computer, for taking ownership of files created on other systems where you - for whatever reason - did not set the permissions correctly. Your home computer should be configured using one of the above methods to automatically create files with the right permissions, therefore you'll never need to chown a file created by your computer while using the drive with a third-party computer.

Don't get me wrong, here: if you find FAT32 does what you need - and especially if you're transferring files between Windows and other operating systems - then continue to use it. I'm just correcting misconceptions so that anyone coming across this thread via a search doesn't see it and say "oh, well, obviously I can't use ext4 for a removable drive" - something which is clearly mistaken. Any ext format works fine on removable drives, so long as you know what you're doing with permissions - and has the advantage that you can transport files larger than 4GB, something that FAT32 won't let you do.
lp rob1 17th November 2012, 11:55 Quote
Setting a umask on my home computer will only fix the problems there - no where else. Removable drives are portable by nature, so a solution that works only in certain places is not a solution to me.
And yes - I do have root access on my home computer, but this will only let me set the mount options on my home computer, as far as I am aware. On any other computer the mount options will just revert to the defaults, and the cycle begins again. Unless there is something I am missing here?

I very often hit the 4GB file limit on FAT32. I usually get around this temporarily by creating an ext4 partition on my USB drive, manually making sure the permissions are correct for my files, then removing the partition once the work is done. I really do want to use ext4, but until changing set permissions is a completely transparent process, I simply cannot use it effectively.
I could set the umask on every computer that I use, or I could manually change the permissions every time I add new files in, but I won't because this isn't transparent anymore. I cannot change the mount options on every computer I use, simply because I don't have (legitimate) root access to them. A solution needs to be something that I can apply once to the ext4 partition, and have it permanently set to fully permissive permissions no matter which computer I use it on.
Gareth Halfacree 17th November 2012, 14:09 Quote
Quote:
Originally Posted by lp rob1
Setting a umask on my home computer will only fix the problems there - no where else. Removable drives are portable by nature, so a solution that works only in certain places is not a solution to me..
I'm aware that portable drives are portable - but you're missing my point entirely.

Right, let's go for scenarios. The way I see it - and feel free to correct me if I'm wrong - there are two scenarios at play here: you need to access a file you created on your home computer on a third-party computer, and you need to access a file you created on a third-party computer on your home computer. Let's call those Scenario 1 and 2 respectively.

Scenario 1 works invisibly: set the create mask on your home computer - mount option in fstab, mount option passed through FUSE, umask in /etc/profile, umask in ~/.bashrc, feel free to pick any one of those multiple options - and any files you create on your drive will be set up so any user account can access them. Plug the drive into the third-party computer, and the files are immediately accessible with no changes required.

Scenario 2 is, if I'm reading your post right, where you're getting confused. You're quite right that, if you make no changes to the third-party computer, the files will be created with the system-wide mask of 002 or 022 (depending on the system in question.) As a result, you take the drive back to your computer and receive a permissions error when you try to modify the file.

But remember what I said about only needing root access on your home computer? Take ownership of the file you're trying to open with "sudo chown yourusername filename" - or, if your file manager supports it, by right-clicking on the file and going into the properties menu. Et voila: the file is now fully accessible on your home computer. Want to make it so it's the same permissions as every other file on the drive? "chmod 666 filename" - boom: it's now set so that all users, regardless of UID or GID, have read and write access, and will remain set that way on every computer to which the drive is connected.

Want to make things easier on yourself? For computers where you have a user account but no root access, put "umask 000" in your ~/.bashrc - you don't need any special permissions to do this. That'll set the thing up so it's identical to your home computer, i.e. it all just works invisibly without you having to do anything further. If you're using a guest system where you don't have a permanent account, put the files in a directory labelled TRANSFER - this will remind you to take ownership of them and set the permissions correctly when you get home, after which you can move them to any other directory on the drive.

So. we've now got three scenarios: Scenario 1, transferring from your home computer, is solved; Scenario 2, transferring from a computer on which you have a permanent account, is also solved; Scenario 3, transferring from a computer on which you have no permanent account, requires an extra step that would not be necessary if you were using FAT32, but I would still argue is 'solved' in that you need remember to do nothing but take ownership when you get home - something the file system will remind you of when you try to modify the file and get a permissions error.

Scenarios 1 and 2 are functionally identical to using a permissionless file system like FAT32: once configured, everything just works invisibly with no input required from you. Scenario 3, I'll grant you, is not identical to a permissionless file system - but it's hardly a chore, either. Remember, too, that the default permissions mask is set to allow read access to all: if you find yourself on a computer with no root access and with files owned by a differen GID, you can just copy the file: read access will let you do that, and the new file will have read and write access for your current UID and GID (on most systems: those set to 022 wil have read-only group access, but you can modify that if you want.)

Like I say, though: if you're happy using FAT32, use FAT32 - but don't tell people it's because ext4 is rubbish and doesn't work with external drives, because that ain't true in any way, shape or form. Me, I prefer being able to easily transfer files greater than 4GB - if I'm shuffling files, they're usually large video files, backup archives or disk images - so I use ext4 for all my external drives and NTFS for the ones I use when transferring to or from Windows.
lp rob1 17th November 2012, 16:19 Quote
Hmm, but it is Scenario 4 which creates problems - transferring files between two computers, with different UIDs, and without root access. For example, transferring between my school account (UID 20106) and a friend's school account (UID 20105) would render the files unread/writable if I forget to set permissions correctly, which in itself is hardly transparent. And I cannot use bashrc because my 'home' is refreshed every boot of the school computers, then my real home area is mounted in a subdirectory of that, so any changes would not be persistent.

I never said that that ext4 is rubbish for external drives - I am simply expressing my opinion on the subject and presenting my experiences. I would much prefer to be using ext4 over FAT32, mainly for the size limit reason, but I cannot justify it if I have to mess around with permissions every time I write a file, or have the chance that I cannot read my files if I forget that step.
Gareth Halfacree 17th November 2012, 16:55 Quote
Quote:
Originally Posted by lp rob1
Hmm, but it is Scenario 4 which creates problems - transferring files between two computers, with different UIDs, and without root access. For example, transferring between my school account (UID 20106) and a friend's school account (UID 20105) would render the files unread/writable if I forget to set permissions correctly, which in itself is hardly transparent.
It shouldn't do: default permissions for new files on Linux are either read/write for user and group and read-only for all or read-write for user and read-only for everyone else. As a result, any file you create on your external drive should be at least readable, if not writable, by your friend - is this not the case?
Quote:
Originally Posted by lp rob1
And I cannot use bashrc because my 'home' is refreshed every boot of the school computers, then my real home area is mounted in a subdirectory of that, so any changes would not be persistent.
That's a bleedin' silly setup. Can you not ask an admin to configure your .bashrc for you?
Quote:
Originally Posted by lp rob1
I never said that that ext4 is rubbish for external drives - I am simply expressing my opinion on the subject and presenting my experiences.
Which is fair enough - but you phrased it as a problem with permissions on file systems in general, not a problem with permissions on file systems on a very specific scenario you've experienced.
Quote:
Originally Posted by lp rob1
I would much prefer to be using ext4 over FAT32, mainly for the size limit reason, but I cannot justify it if I have to mess around with permissions every time I write a file, or have the chance that I cannot read my files if I forget that step.
There should be absolutely no chance that you won't be able to read your files.

To satisfy my curiosity, could you - when you get a chance - create a new file at school and tell me the permissions ("ls -larth filename")? I'm really curious as to how this is set up, because I've yet to see a Linux system that defaults to creating files only readable by the original owner - which is the scenario you're describing there.
lp rob1 17th November 2012, 18:52 Quote
Perhaps I phrased my reply badly. I understand that files will be readable with the default umask, but I still want permissions to be effectively 777 when new files are created, and I cannot rely on system specific options like umask or FUSE options. Besides, putting umask into bashrc sets the umask for all new files created, so you get a security problem for files that you are saving elsewhere, like in a home area or /tmp.
I am trying to aim for the transparency of permissions given with FAT32 - aka no effective permissions at all. I could simulate it by creating a folder with 777 permissions on my memory stick, then ending up with an equivalent of the /tmp directory on my USB drive (as in - the owner can edit, but others only have read only) but this isn't quite a complete solution. Despite this, this thread has fixed some of my gripes with ext4 on a removable drive, so I might try for a more complete solution at a later date. Thanks, Gareth. ;)
Gareth Halfacree 19th November 2012, 07:13 Quote
Quote:
Originally Posted by lp rob1
Despite this, this thread has fixed some of my gripes with ext4 on a removable drive, so I might try for a more complete solution at a later date. Thanks, Gareth. ;)
No worries - glad to help!
Log in

You are not logged in, please login with your forum account below. If you don't already have an account please register to start contributing.



Discuss in the forums