'helpful' AI

[artfizz]

I asked Google Gemini about the format of CASE statements in BBC BASIC, and it told me I could do this:

Code: Select all

WHEN value1 TO value2: do something

so I did that:

Code: Select all

10 ascii_value = 50
20 CASE ascii_value OF
30 WHEN 0 TO 31: PRINT "unprintable"
40 WHEN 48 TO 57: PRINT "numeric"
50 ENDCASE

but I got 'Syntax error at line 40' with the cursor on the "TO".
I was puzzled why there was no syntax error on line 30.
It took me a while to spot the culprit!

I asked Gemini again in a new session, and it gave a different answer:

Code: Select all

WHEN value1, value 2
WHEN value3

The moral to the story - according to the AI - is: 'AI is a tool, not a replacement for expertise'.

[Richard Russell]

I asked Gemini again in a new session, and it gave a different answer:

AIs deliberately introduce a random element, and that may well improve their subjective performance in language and reasoning tasks, but it's at best confusing when they sometimes produce correct code and sometimes incorrect code.

The underlying problem here, I suspect, is that there are insufficient BBC BASIC programs in the AI's training data. An example of 'unintended consequences' is the fact that most programs are in a binary 'tokenised' format that probably can't be used for training.

Maybe if BBC BASIC had used plain-text format for its programs, by default, the AIs would work better.

[Richard Russell]

The moral to the story - according to the AI - is: 'AI is a tool, not a replacement for expertise'.

Something else to bear in mind is that you can improve the AI's "expertise" by uploading the BBC BASIC manual (most AIs have a 'paperclip' button for this) before you enter the prompt. You could potentially upload the entire BBC BASIC for Windows manual in the form of the PDF; there isn't a PDF of the BBC BASIC for SDL 2.0 manual but you could upload the individual HTML files.

I've not tried this myself but in principle it should improve the AI's performance at BBC BASIC coding.