Drop trailing spaces from files in normalise on Windows

[ndmitchell]

writeFile "foo    " ""
writeFile "bar" ""
doesFileExist "foo" -- True
doesFileExist "foo   " -- True
doesFileExist "bar   " -- True

Seemingly any trailing spaces in a path are ignored by Windows. Note that directories behave the same way if they are the last argument, but if you add a path separator, then they don't get their spaces removed. Very weird.

[codygman]

Is this supposed to be for the directory library?

http://hackage.haskell.org/package/directory-1.2.4.0/docs/src/System-Directory.html#doesFileExist

I can verify this bug still exists.

[ndmitchell]

The directory library is probably using the underlying Windows API calls, so I would need to first verify that the native Win32 API functions have the same behaviour. If not, it's a directory bug, if it is, then it's a filepath bug.

[codygman]

Yeah, I was trying to figure out how withFileStatus worked yesterday to test that:

(withFileStatus "doesFileExist" name $ \st -> do b <- isDirectory st; return (not b))

So it doesn't seem to use the win32 library at least.

Then the definition of withFileStatus:

withFileStatus :: String -> FilePath -> (Ptr CStat -> IO a) -> IO a
withFileStatus loc name f = do
  modifyIOError (`ioeSetFileName` name) $
    allocaBytes sizeof_stat $ \p ->
      withFilePath (fileNameEndClean name) $ \s -> do
        throwErrnoIfMinus1Retry_ loc (c_stat s p)
        f p

I suppose we could strip the filename of leading/trailing spaces where fileNameEndClean is done. I'm hesitant to do it since I don't quite get the purpose of withFileStatus nor how it works.

[ndmitchell]

I think Haskell should be following the underlying Windows NTFS semantics, so the first question is what are they.

[codygman]

Do not end a file or directory name with a space or a period. Although the underlying file system may support such names, the Windows shell and user interface does not. However, it is acceptable to specify a period as the first character of a name. For example, ".temp".

source: https://msdn.microsoft.com/en-us/library/aa365247%28VS.85%29.aspx#file_and_directory_names

So NTFS supports it but the shell and user interface do not. So now I'm not sure which one is correct, though I'm leaning towards trailing spaces being invalid since Windows itself doesn't support them.

[codygman]

I guess the question is:

Does this function test whether the file or folder exists on the file system or in a usable state from Windows?

Whatever the case we need to make sure that createFile can only create files valid as per doesFileExist.

[codygman]

Also found this:

The Win32 subsystem enforces additional constraints on legal file names. The name can be at most MAX_PATH characters long (defined in windef.h as 260 characters), may not have trailing dots or spaces, and file names are case preserving, not case sensitive — if two files exists with names that differ only in case, you will only be able to manipulate one of them through Win32 APIs.

http://blogs.msdn.com/b/brian_dewey/archive/2004/01/19/60263.aspx

[codygman]

As a last little bit of evidence it looks like Windows Server 2012 automatically strips filenames with leading/trailing spaces.

https://support.microsoft.com/en-us/kb/2829981

[ndmitchell]

So all this leads me to the conclusion that normalise should drop trailing spaces on Windows, but only literal space, not any isSpace characters. The idea behind normalise is it produces files that are "equivalent to" the file you gave it, and since spaces are not relevant when saving or testing, they can be dropped.

[ndmitchell]

Windows is weird. So experimenting with the underlying Windows API functions, it appears the API strips any trailing . or space characters first, before operating on it. This means it isn't a semantic operation. Specifically, creating foo produces foo, but creating foo./ produces foo.. You can have a valid directory name which has trailing spaces/dots, but it can't be passed to the creation functions as the end. If I were to normalise trailing spaces/dots, then we'd lose the property:

normalise x </> normalise y = normalise (x </> y)

Therefore, I'm inclined to close this bug and change nothing. Other than observing that Windows remains weird.

[ndmitchell]

@codygman - I'd be keen to see if you share my final viewpoint, or if you think there is more to do here.

[codygman]

That depends on if windows compatibility is a major goal or not.

Will comment further after work.
On Dec 22, 2015 8:23 AM, "Neil Mitchell" notifications@github.com wrote:

@codygman https://github.com/codygman - I'd be keen to see if you share
my final viewpoint, or if you think there is more to do here.

—
Reply to this email directly or view it on GitHub
#48 (comment).

[ndmitchell]

Windows compatibility has always been and remains an essential requirement.