BBC BASIC Programmers' Reference

Accessing huge files

by Richard Russell, October 2007

Most of the BBC BASIC for Windows file-handling statements and functions are fully compatible with files in excess of 2 Gbytes (2^31 bytes) in length. You can successfully open (using OPENIN or OPENUP) files of any length, and files created with OPENOUT can exceed 2 Gbytes in length if you write enough data to them! Sequential reads (using INPUT#, BGET# and GET$#) and writes (using PRINT# or BPUT#) will work correctly whatever the file size, even when crossing the 2 Gbytes 'boundary'.

The only exceptions are EOF#, PTR# and EXT#. When setting the file pointer using PTR#file= you can only set it within the first 2 Gbytes of the file, and when setting a file's length using EXT#file= you can only set it to a maximum of 2^31-1 bytes. Using PTR# to read the current file pointer will work correctly only if the pointer is within the first 2 Gbytes, and you cannot read the file's size using EXT# if it exceeds 2 Gbytes. The EOF# function will not work reliably if the file length exceeds 2 Gbytes.

Should you need to position the file pointer beyond the first 2 Gbytes, or to read the length of a file bigger than 2 Gbytes, you can use the PROCptr and FNext routines listed below. Note that the pointer value passed to PROCptr and the file size returned from FNext necessarily exceed the range of an integer variable, so you should use variant variables to hold these values. If you need the values to be precise (which you probably will, at least in the case of PROCptr) you must run your program in *FLOAT 64 mode.

Here is the replacement for EXT#:

        DEF FNext(F%)
        LOCAL H%, L%
        SYS "GetFileSize", @hfile%(F%), ^H% TO L%
        = H% * 2^32 + FNuint(L%)

Note that this function may not return the correct value while you are writing to the file; it should normally be used only when the file has been opened for reading.

Here is the replacement for PTR#:

        DEF PROCptr(F%, p)
        LOCAL H%, L%
        PTR#F% = 0
        H% = p / 2^32
        L% = FNintu(p - H%*2^32)
        SYS "SetFilePointer", @hfile%(F%), L%, ^H%, 0
        ENDPROC

It is not straightforward to read the file pointer (beyond 2 Gbytes) or to write the file's length (in excess of 2 Gbytes) so routines to do that are not provided. It will normally be possible to avoid these operations.

The FNuint and FNintu functions are listed in the article Handling unsigned 32-bit values but are reproduced below for convenience:

        DEF FNuint(N%) = (N% >>> 1)*2 + (N% AND 1)

        DEF FNintu(N) = ((N / 2) << 1) - (INT(N / 2) <> N / 2)