Standard Deviation challenge

[Hated Moron]

In another recent thread, reference was made to the suitability of BBC BASIC for mathematical operations. Coincidentally, a couple of days ago, I received an enquiry about operations on lists of numbers: mean, median, mode and so on.

I responded by listing the standard BBC BASIC routines for these, but I also added the code for Standard Deviation, which I've always thought is a particularly nice demonstration of the language's mathematical capabilities.

But then I thought: is that code as well-known as it should be? If I had not been around to provide it (which I won't be for much longer) would the answer have been forthcoming if he asked here, for example?

So here's the challenge. Given a numeric array of arbitrary size, array(), list the code to calculate the Standard Deviation of all the values in the array in no more than two BASIC statements! No cheating, no libraries, just standard BBC BASIC code.

[DDRM]

Intriguing! There isn't a SUM(array) command that sums the elements, is there? I could do it then... The MOD(array) allows us to get the sum of squares...

Best wishes,

D

[hellomike]

Of course there is a SUM() function. It has been there for ages. See the manual.
SUM() function.

[Edja]

Code: Select all

   DIM array(10):array()=0,1,2,3,4,5,6,7,8,9,10
   array()=array()-SUM(array())/DIM (array(),1)+1
   stdev=MOD(array())/SQR(DIM (array(),1)+1)

This results in a stdev= 3.2016.
However the calculation I did with the Excel-function STDEV (as a check) gives 3.3166. I'm still trying to figure out where I went wrong in my code (I suppose Microsoft knows how the calculate a standard deviation).

Edja

[Edja]

Code: Select all

      DIM array(10):array()=0,1,2,3,4,5,6,7,8,9,10
      array()=array()-SUM(array())/(DIM (array(),1)+1)
      stdev=MOD(array())/SQR(DIM (array(),1)+1)

I've added some brackets in the second line that were missing.
It now produces a stdev=3.1623, still different form Excel's 3.3166. Still searching for the error....
Edja

[DDRM]

HelloMike: yes, of course - thanks. I went straight to the section on array arithmetic, which didn't mention it!

Edja: I suspect you are calculating the population standard deviation (since you are using all the data), and Excel may be returning the sample standard deviation? But in your second line shouldn't the DIM (array(),1)+1 be bracketed (as it is in the third line): since you are calculating the mean value you need to divide by 11 (to allow for the array being 0-based)?
Ironically, if you bracket the +1 on line 2, and remove it on line 3 (so you divide by n-1, as required for the sample SD), you DO get the same answer as Excel...

Best wishes,

D

[Edja]

Hi DDRM ,

Our replies must have crossed. I've already made the correction of the missing brackets.
Yes, you're right. If I now remove the +1 in the last line then the result matches the Excel function STDEV.S (sample standard deviation).
My code was aiming for the total population standard deviation in which case the +1 (11 values in total) is required. To do this Excel offers STDEV.P and that results in 3.1623, matching my result.
I should have used the right formula (STDEV.P or STDEV.S) to check my code.
I never was any good in statistics.

Regards,
Edja

[Hated Moron]

Code: Select all

      DIM array(10):array()=0,1,2,3,4,5,6,7,8,9,10
      array()=array()-SUM(array())/(DIM (array(),1)+1)
      stdev=MOD(array())/SQR(DIM (array(),1)+1)

Yes, well done! There is a small simplification that you missed:

Code: Select all

      DIM array(10)
      array()=0,1,2,3,4,5,6,7,8,9,10
      array() -= SUM(array())/(DIM (array(),1)+1)
      stdev=MOD(array())/SQR(DIM (array(),1)+1)

[Hated Moron]

If I now remove the +1 in the last line then the result matches the Excel function STDEV.S (sample standard deviation).

But it makes the code wrong! Your original is correct (the +1 is required). Should I now take back the (non-existent) prize?!

[DDRM]

One other comment I'd make about that solution is that it destroys the contents of the array, which may be undesireable. It might be better to accept the "extra cost" of one more line (or a slightly more complicated formula) to avoid that in a working version...

This solution assumes that the array is being used 0-based, and has no "empty" entries ,e.g. at the top, (which would be 0s in a numeric array) so that needs stating as a requirement.

Best wishes,

D