Shared file access: FTell(), FEof() and all RDD record positioning use the wrong file offset on .NET 6 and later

[hpetriffer]

Shared file access: FTell(), FEof() and all RDD record positioning use the wrong file offset on .NET 6 and later

Summary

XsWin32FileStream never overrides Position. Since the .NET 6 FileStream rewrite that makes FTell(), FEof(), FReadLine() and — worse — the whole RDD record positioning path (SafeReadAt() / SafeWriteAt() / SafeSetPos()) read and write at the wrong file offset, silently and without an exception, for every file opened with shared access.

XsWin32FileStream does all of its I/O with the Win32 API on the OS file pointer (SetFilePointerEx + ReadFile / WriteFile), but Position falls through to the base class. On .NET Framework that worked: touching SafeFileHandle set _exposedHandle, and FileStream then re-synced its cached _pos from the OS handle on every access (VerifyOSHandlePosition). The .NET 6 FileStream strategy rewrite removed that re-sync, so FileStream.Position is now a purely in-memory counter that the overrides in XsWin32FileStream never touch. It stays at 0 forever while the real file pointer moves.

Affected

Any file opened with share != FileShare.None — that is FO_SHARED, FO_DENYWRITE or FO_DENYREAD — on Windows. That is the branch in XsFileStream.CreateFileStream() which returns CreateWin32FileStream(). It includes the DBF, memo and index files opened by the RDDs.

Runtime	Result
.NET Framework 4.8	OK
.NET Core 3.1	OK
.NET 5	OK
.NET 6	broken
.NET 8	broken
.NET 10	broken

So this is not a .NET 10 regression. It started with .NET 6 and is still present in .NET 10.

Reproduction

FUNCTION Start() AS VOID
    LOCAL cFile := "fstest.txt" AS STRING
    MemoWrit(cFile, "Hello World line1" + CRLF + "second line here" + CRLF)

    VAR h := FOpen(cFile, FO_READWRITE + FO_SHARED)   // -> XsWin32FileStream

    FSeek3(h, 6, FS_SET)
    ? FTell(h)                  // expected 6           .NET 6+: 0
    ? FReadStr(h, 5)            // expected "World"     ok
    ? FTell(h)                  // expected 11          .NET 6+: 0

    FSeek3(h, 0, FS_SET)
    ? FReadLine(h, 100)         // expected "Hello World line1"   ok
    ? FReadLine(h, 100)         // expected "second line here"    .NET 6+: ""
    ? FEof(h)                   // expected TRUE at end           .NET 6+: FALSE

    FClose(h)
    RETURN

Measured against the same XSharp.Core sources, only the host runtime differs:

.NET Framework 4.8                     .NET 10.0.11
OK   FTell after FSeek(6)  = 6         FAIL FTell after FSeek(6)  = 0
OK   FTell after FRead     = 11        FAIL FTell after FRead     = 0
OK   FReadLine #2 = "second line here" FAIL FReadLine #2 = ""
OK   FEof at end  = TRUE               FAIL FEof at end  = FALSE

The serious part: silent data corruption in the RDDs

SafeReadAt(), SafeWriteAt() and SafeSetPos() in FileStreamExensions position by assigning oStream:Position and then calling Read() / Write(). That is how the entire RDD layer addresses records (DbtMemo, FlexArea, DBF headers). With a broken Position setter the assignment updates only the base class counter, while Read() / Write() still operate at the untouched OS file pointer:

.NET Framework 4.8                      .NET 10.0.11
OK   SafeReadAt(6,5) = "World"          FAIL SafeReadAt(6,5) = "Hello"
OK   SafeSetPos(0)+SafeRead = "Hello"   FAIL SafeSetPos(0)+SafeRead = " Worl"

No exception is raised. Reads and writes simply land at the wrong offset.

Further defects found in the same class

All in src/Runtime/XSharp.Core/Types/SharedFileStream.prg:

1. ReadByte() is not overridden. .NET Core's FileStream overrides it and goes straight to its own strategy, bypassing the Read() override, so it returns the byte from the wrong position. The same applies to Read(Span<byte>), Write(ReadOnlySpan<byte>) and the async overloads, which .NET Core also overrides.
2. Read() with offset != 0 copies count bytes instead of the number of bytes actually read. On a short read that overwrites bytes in the caller's buffer past the end of the file with zeroes from the temporary buffer, and it throws when the target buffer is only large enough for offset + bytesRead.
3. Read() returns -1 on failure, which violates the Stream.Read contract (the result must be between 0 and count) and breaks generic consumers such as Stream.CopyTo and StreamReader.
4. The base stream is constructed with a 64 KB buffer (bufferSize 0xFFFF). On .NET 6+ that wraps it in a BufferedFileStreamStrategy. The overrides never use that buffer, but every code path that is not overridden reads through it and therefore sees stale data, and Flush(lCommit) never calls SUPER:Flush(), so anything sitting in the buffer is silently dropped. A stream for shared access should not be buffered at all.

[hpetriffer]

@RobertvanderHulst Is there a reason why the implementation doesn't use the managed FileStream everywhere? If desired, I can submit a PR.

[RobertvanderHulst]

@hpetriffer There was a problem with shared file access and speed. Therefore we have overwritten some of the methods to improve the speed. We'll look into the issues that you have reported and will fix them. Most likely we will not use our own filestream class at all for .Net 6 and later (just like we're not using our own class on not-windows platforms).

[RobertvanderHulst]

@RobertvanderHulst Is there a reason why the implementation doesn't use the managed FileStream everywhere? If desired, I can submit a PR.

If you have working code, then please send a PR

[hpetriffer]

@RobertvanderHulst I sent you an email with a few questions about the previous performance issues. As soon as I receive a reply, I'll be able to submit a PR.