Saturday, April 24, 2010

NTFS junctions are a poor substitute for symbolic links

Recently, I have been working in a Windoze-XP environment using multiple sandboxes checked out from CVS. Each sandbox needs a large collection of third party libraries (headers, lib and .dll files). Each third party directory takes up around 2GB. This space very quickly mounts up. In Unix we would use symbolic links to make all the third party directories point to a common place. But what to do in Windoze? Some people would say that NTFS junctions is the answer. Well, they can help and I am using them for this purpose but there are some nasties and I am documenting them here. I once saw a signature on USENET which goes: "Those who do not understand Unix are condemned to reinvent it, poorly." This is certainly true of Windoze. It doesn't understand symbolic links so it is no suprise to learn that NTFS junctions are a very poor imitation. Here are several points to bear in mind:
  • They are a cross between hard links and symbolic links. Like the hard link, they can onloy refer to files on the same partition, but like symbolic links the path of the link is stored which means it can become invalid. This is because Windoze filesystems do not support links! Also, only directories can be linked.
  • Windoze-XP has insufficient support for NTFS junctions despite them first appearing in Windoze-2000. There are no commands to manipulate them. You will have to download and install the SysInternals commands and use the junction command.
  • More explicit support for NTFS junctions was added to Vista and is also in Windows 7. But even by these versions of Windoze, support is incomplete. When you use Windows Explorer to navigate to a directory that contains NTFS junctions, if you delete the directory it deletes the files that the junctions refer to!!! See for the gory details.
The only way to delete a junction safely is to the use the SysInternals junction command. There is no safe way to delete a directory that may contain junctions. Currently, I do not know what the solution is to these problems. The cygwin rm command does not help, it follows the junction references just like Windoze Explorer does. I have a feeling a special delete command may have to be written. I will probably have to write it, in Python, and it will probably use the SysInternals junction command to delete any NTFS junctions it finds.


Andrew Marlow said...

I will probably only use NTFS junctions privately. I have decided that I cannot advocate them for general use on a project. They are just too dangerous, given the insufficient support by Microsoft.

Basically, when one uses NTFS junctions, any standard delete command becomes dangerous. The delete command will follow junctions and delete what the junction is pointing to.

Claus said...

Thanks for the blog post! I just created some junctions using Microsoft's linkd utility (from the Windows Resource Kit), then tried to delete them with Cygwin's "rm".

This actually worked quite well. Both "rm" and "rm -r" only removed the junction, not the directory which the junction was referring to.

Maybe there were some recent improvements? My tests were on Cygwin 1.7.5.

Andrew Marlow said...

What you saw may have been an optical illusion. The weird behaviour of delete in the presence of junctions includes the fact that it marks the junction for delete but the delete is delayed until the recycle bin is emptied. Nasty, eh?

Claus said...

Are you saying that Cygwin's "rm" command moves stuff to the Windows recycle bin? Over all those years with Cygwin, I deleted many thousands of files using "rm", so I guess I would have noticed if at least some of them ever showed up in the recycle bin.

Andrew Marlow said...

That's what I heard it did. And it did seem that way to me, but I could be wrong. However, whether cygwin does it right or not, Windows does it wrong. If you use Windoze explorer to delete a dir that contains NTFS junctions it will do the wrong thing. Those who do not understand Unix are condemned to reinvent it poorly.

Andrew Marlow said...

I am grateful (sort-of) to the scumbag who polluted my blog with comment spam. This has forced me to find out how to enable comment moderation. In changing this flag I have also opened up the blog so that anyone can comment rather than them having to be from a googlemail account. But comments won't be posted now until they get moderation approval from me.

Anonymous said...

Not only did Google want me to sign in, but it wanted me to sign up for a Blogger page. I canceled and hope you got my post about how to delete a Junction. If not, see which has the basic idea I used.


Andrew Marlow said...

I was put off by the bit where it said:

"It's also best to do this on a clean install of Windows, unless you don't mind waiting awhile..."

and the webpage requires javascript in order to render at all. Still, thanks for the info...