Windows 11 VT100 emulation

[Hated Moron]

I'm experiencing a problem running the Console Mode edition of BBC BASIC in Windows 11.

All the Console Mode editions assume a VT100 (ANSI) compatible terminal, which the default Windows 11 shell (Command Prompt) certainly is. But what I'm finding is that occasionally the 'query cursor position' escape sequence is not receiving a valid response from the terminal.

The practical upshot of this is that, unpredictably, a program (or command like *DUMP) will suddenly freeze for a second or so and then abort with an Escape error. As you can imagine, this is quite annoying and inconvenient!

This behaviour seems quite specific to Windows 11. I've not experienced it with any of the other terminals I've tried (Windows 10 console, PuTTY in Windows, and the default terminals in MacOS, Linux and Raspberry Pi).

If you are running Windows 11 can you please try this yourself? Run the Console Mode edition of BBC BASIC and, for example, issue a *DUMP command with a relatively large file. Does it dump the entire contents of the file without aborting with an Escape error?

Unless this issue is unique to this laptop, which seems unlikely, we may have to try to find a workaround.

[Hated Moron]

On 24/07/2023 22:59, Storer, Darren wrote (cross-posted from the discussion group):

Yes, I can reproduce the problem here with Windows 11 (i7, fast GPU, SSD, 32 GB RAM etc.). On different runs, "Escape" was displayed at varying points - there was no discernable pattern. Under PowerShell, the same command seemed to run for a little longer but always drops out to "Escape".

So do you have the time / resources / inclination to help?

I presume it's probably a bug in Microsoft's latest VT100 emulation (if it was a shortcoming at the BBC BASIC end surely it would have shown up earlier with one of the other terminals) so we are probably looking at trying to find a workaround, although I suspect that won't be easy.

[Hated Moron]

we are probably looking at trying to find a workaround, although I suspect that won't be easy.

More in hope than expectation, I made the simplest possible change I could think of that might conceivably help: I increased the length of the input buffer (in case the Windows 11 VT100 emulation outputs some longer escape sequences than the others do).

Unless it's purely a coincidence, which is possible, it actually seems to have helped! Can you download the new version (0.44) and try it please?

[Hated Moron]

On 25/07/2023 16:31, J.G.Harston wrote (cross-posted from the discussion group):

This is something that's had me tearing my hair out at times reading ANSI keyboard input. Certain sequences are terminated by "no key"... but how do you know there's "no key" or "slightly delayed key"?

In a sense it's Acorn's fault for using Escape as a 'user abort' key because it fundamentally conflicts with the use of that code to introduce an 'escape sequence', which presumably was an established convention when the BBC Micro was developed.

Clearly their system was waiting up to 1sec after <esc> to decide there was no following character, which is a working solution, but 1sec seems overkill.

Any distinction based on timing (and as you say there's no other way) is going to be unreliable. If the link to the terminal is over a network connection, which could be half-way across the world, there's no delay that is guaranteed to be 'enough'.

If I remember rightly I have tried to compromise in the BBC BASIC Console Mode editions by setting the delay to 1000 ms when a cursor query command has been sent and therefore an escape sequence response is expected, but to only 200 ms if not. Not foolproof, but better than a fixed delay for all circumstances.

[Hated Moron]

On 25/07/2023 16:39, J.G.Harston wrote (cross-posted from the discussion group):

The code I eventually worked down to was:

I should emphasise that the Windows 11 problem I reported in this thread is not related to the issue you are discussing (how to distinguish pressing the Escape key from an escape sequence) so strictly speaking this is off-topic.

It seems that the likely cause of the problem I reported was that the input buffer wasn't large enough, possibly because the Windows 11 terminal emulation is sending escape sequences longer than I expected, or perhaps two or more escape sequences concatenated. This caused the buffer to overflow and consequently the escape sequence to be truncated or corrupted.

Code: Select all

fflush(stdout);
read(STDIN_FILENO, &ch, 1);        /* Read without flushing    */
ch=ch & 0xFF;
if (ch != 27)     return ch;        /* Not <esc>            */
if (kbhit() == 0) return ch;        /* Nothing pending        */

That seems fundamentally unreliable to me because (if I've understood the code correctly) you are relying on the second character of the escape sequence already having arrived as soon as you read the escape code itself. That may well commonly be the case with a local connection on the same machine, but I wouldn't expect it to be with a serial link or network connection.

An explicit delay is essential, in my opinion.

[Hated Moron]

On 25/07/2023 23:17, J.G.Harston wrote (cross-posted from the discussion group):

Yes, it "felt" wrong when I got that code working. It was 2018 when I wrote that bit of code, and it took a lot of digging around Microsoft's developer pages to eventually find some recommended code. I can only speculate that buried within Windows it has already read a full two-byte sequence from the keyboard, which has an unambigous length, and has pre-queued the ANSI translation of it.

Entirely likely when the keyboard (terminal) and client are on the same machine, but since the BBC BASIC Console Mode editions use STDIN and STDOUT as the communication channel they are designed to be usable with all sorts of connections, whether local or across a serial link or a USB cable or a local-area-network or even the internet.

In all the 'non local' cases there's no guarantee that the group of characters comprising an escape sequence will even be in the same packet, let alone that they will arrive 'simultaneously'. The only relatively safe approach is to use quite a long delay; I've always found one second to be adequate but I bet there are situations when slow or unreliable connectivity means even that isn't sufficient.

[Hated Moron]

On 25/07/2023 18:35, Storer, Darren wrote (cross-posted from the discussion group):

Good news! Using a lower specification PC (still Windows 11 and i7 etc.), I cannot reproduce the problem using the newly released version.

Once I'm home, I'll re-test using the same platform as yesterday.

Thanks, I await your report with interest.

[JeremyNicoll]

The only relatively safe approach is to use quite a long delay; I've always found one second to be adequate but I bet there are situations when slow or unreliable connectivity means even that isn't sufficient.

I'm sure you're right about 1 second not always being long enough. Consider for example when traffic is carried according to RFC 1149:
https://datatracker.ietf.org/doc/html/rfc1149

The two updates to that RFC, ie RFC 2549 and 6214 are amusing too. I particularly like the extra detail and ASCII art in 2549.