Differences

This shows you the differences between two versions of the page.

--- inputting_20real-time_20audio [2018/03/31 13:19] – external edit 127.0.0.1
+++ inputting_20real-time_20audio [2024/04/19 17:43] (current) – richardrussell
@@ Line 1: / Line 1: @@
 =====Inputting real-time audio=====
-//by Richard Russell, November 2006//\\ \\  Most PCs provide the capability of inputting audio, typically either from a microphone or from a line-level input. This article describes how a BBC BASIC for Windows program can read and process audio data from this source in real-time (i.e. without it having to be recorded to a file first).\\ \\
+Most PCs provide the capability of inputting audio, typically either from a microphone or from a line-level input. This article describes how a //BBC BASIC for SDL 2.0// or //BBC BASIC for Windows// program can read and process audio data from this source in real-time (i.e. without it having to be recorded to a file first).
+===== BBC BASIC for SDL 2.0 =====
+//by Richard Russell, April 2024//
 ==== Preliminaries ====
-\\  As is usual for programs accessing the Windows API, it is important to trap errors, and closing the window, so that the necessary 'cleanup' operations can take place:\\ \\
+As is usual for programs accessing the SDL 2.0 API, it may be important to trap errors, or closing the window, so that the necessary 'cleanup' operations can take place:
+<code bb4w>
+      ON ERROR PROCcleanup : MODE 7 : PRINT REPORT$ : END
+      ON CLOSE PROCcleanup : QUIT
+</code>
+The **PROCcleanup** routine is listed later. You may want to change the error reporting to a different method.
+==== Selecting the input device ====
+Typically the host system will have multiple audio input devices, which may include a **microphone** or a **line input** or a **Stereo Mix** etc.  Your program must select which of these devices it wishes to input audio from, and the easiest way is probably to present a list to the user to choose from:
+<code bb4w>
+      REM. Find number of audio capture devices:
+      SYS "SDL_GetNumAudioDevices", 1, @memhdc% TO numDevices%
+      IF numDevices% = 0 ERROR 100, "No audio capture devices are available"
+      REM. List the capture device names and put them into an array:
+      DIM deviceName$(numDevices%)
+      PRINT "Available audio capture devices:"'
+      FOR index% = 1 TO numDevices%
+        SYS "SDL_GetAudioDeviceName", index%-1, 1, @memhdc% TO pname%%
+        IF @platform% AND &40 ELSE pname%% = !^pname%%
+        deviceName$(index%) = $$pname%%
+        PRINT ;index% ": " deviceName$(index%)
+      NEXT
+      PRINT
+      REM. Get selected device from user:
+      REPEAT
+        INPUT "Enter device number: " index%
+      UNTIL index% >= 1 AND index% <= numDevices%
+</code>
+You could of course present the choices and accept the selection in different ways, for example a dialogue box containing a list box plus OK and Cancel buttons.  Or you might prefer it to be determined by a configuration file rather than a user selection made each time the program is run.  That's entirely up to you; adapt the above code accordingly.
+==== Choosing the buffer size ====
+The first step is to decide how large the audio buffer should be. To some extent this is an arbitrary decision, but it will depend on things like //latency// (how much time elapses between the audio arriving and it being processed by your program) and the amount of work needed to process the received audio data.\\ \\  It is vitally important that your program can process the audio data quickly enough, otherwise data will be lost, with undesirable results. If there is any variability in the rate at which you can process the data (for example it depends on disk or network accesses) then you may need to use a larger buffer to 'iron out' the fluctuation. In the example below the length of the buffer is 1024 samples; at 44100 Hz stereo that implies a latency of at least 24 milliseconds.
+<code bb4w>
+        SamplesPerBuffer% = 1024
+</code>
+==== Selecting the audio format ====
+The next step is to decide the audio //format// you will use: the principal choices being of sampling rate (the main ones being 11025 Hz, 22050 Hz and 44100 Hz) and number of channels (mono, 1 channel, or stereo, 2 channels). The higher the sampling rate the higher the audio frequency that can be received, but the more work your software needs to do. Normally you should choose the lowest sampling rate suitable for your application, remembering that it needs to be at least **double** the highest audio frequency in which you are interested (according to the [[http://en.wikipedia.org/wiki/Nyquist_rate|Nyquist criterion]]).
+You set up the required audio format and open the audio capture device as follows:
+<code bb4w>
+      DIM want{freq%, format{l&,h&}, channels&, silence&, samples%, size%, callback%%, userdata%%}
+      DIM have{} = want{}
+      want.freq% = 22050
+      want.format.h& = &80 : REM AUDIO_S16LSB
+      want.format.l& = &10 : REM AUDIO_S16LSB
+      want.channels& = 2
+      want.samples% = SamplesPerBuffer%
+</code>
+==== Opening the audio device and creating the buffer ====
+Now the audio capture device can be opened and the buffer created:
+<code bb4w>
+      SYS "SDL_OpenAudioDevice", deviceName$(index%), 1, want{}, have{}, 9, @memhdc% TO Device%
+      IF Device% = 0 ERROR 100, "Couldn't open audio device " + deviceName$(index%)
+      BytesPerBuffer% = have.size%
+      WordsPerBuffer% = BytesPerBuffer% DIV 4
+      DIM Buffer%(WordsPerBuffer% - 1)
+</code>
+This code allows for the possibility that the capture device doesn't support the sampling rate you have chosen and selects a different one.
+==== Starting audio capture ====
+Once you have initialised the audio device using the above code, you can start the real-time capture as follows:
+<code bb4w>
+      SYS "SDL_PauseAudioDevice", Device%, 0, @memhdc%
+</code>
+==== Inputting in real-time ====
+Once the above code has been executed you need to process the received audio buffers fast enough to keep up with the incoming data. The following code constantly cycles, filling the buffer and calling the **PROCprocessbuffer** routine:
+<code bb4w>
+        REPEAT
+          p%% = ^Buffer%(0)
+          R% = BytesPerBuffer%
+          PROCprocessbuffer(p%%, SamplesPerBuffer%)
+          REPEAT
+            SYS "SDL_DequeueAudio", Device%, p%%, R%, @memhdc% TO I%
+            p%% += I% : R% -= I%
+          UNTIL R% <= 0
+        UNTIL FALSE
+        END
+</code>
+In this example the audio processing continues indefinitely, but you can terminate the program prematurely if you wish. If you do, don't forget to execute **PROCcleanup** before exiting the program.
+==== Processing the audio data ====
+Obviously it's only possible to describe this aspect in general terms, because precisely what audio processing takes place will depend on what the program is designed to do. The code below simply calculates the RMS (Root Mean Square) value of the incoming audio:
+<code bb4w>
+        DEF PROCprocessbuffer(B%%, N%)
+        LOCAL I%, V%, sumsq
+        FOR I% = 0 TO N%*2-2 STEP 2
+          V% = B%%!I% AND &FFFF : IF V% >= &8000 V% -= 65536
+          sumsq += V%^2
+        NEXT
+        RMS = SQR(sumsq / N%)
+        ENDPROC
+</code>
+This code is appropriate for monaural input (one channel) where each audio sample consists of a signed 16-bit value in the range -32768 to +32767.
+==== Cleaning up ====
+When you stop the sound capture, or exit the program, you need to shut down the audio input in a controlled fashion:
+<code bb4w>
+      DEF PROCcleanup
+      Device% += 0
+      IF Device% THEN
+        SYS "SDL_PauseAudioDevice", Device%, 1, @memhdc%
+        SYS "SDL_CloseAudioDevice", Device%, @memhdc%
+        Device% = 0
+      ENDIF
+      ENDPROC
+</code>
+This code might form part of a larger routine, if there are other things that need to be shut down.
+===== BBC BASIC for Windows =====
+//by Richard Russell, November 2006//
+==== Preliminaries ====
+As is usual for programs accessing the Windows API, it is important to trap errors, and closing the window, so that the necessary 'cleanup' operations can take place:
+<code bb4w>
         ON ERROR PROCcleanup : SYS "MessageBox", @hwnd%, REPORT$, 0, 48 : QUIT
         ON CLOSE PROCcleanup : QUIT
-The **PROCcleanup** routine is listed later. You may want to change the error reporting to a different method.\\ \\
+</code>
+The **PROCcleanup** routine is listed later. You may want to change the error reporting to a different method.
 ==== Selecting the audio format ====
-\\  The first step is to decide the audio //format// you will use: the principal choices being of sampling rate (the main ones being 11025 Hz, 22050 Hz and 44100 Hz) and number of channels (mono, 1 channel, or stereo, 2 channels). The higher the sampling rate the higher the audio frequency that can be received, but the more work your software needs to do. Normally you should choose the lowest sampling rate suitable for your application, remembering that it needs to be at least **double** the highest audio frequency in which you are interested (according to the [[http://en.wikipedia.org/wiki/Nyquist_rate|Nyquist criterion]]).\\ \\  You set up the required audio format and open the wave input device as follows:\\ \\
+The first step is to decide the audio //format// you will use: the principal choices being of sampling rate (the main ones being 11025 Hz, 22050 Hz and 44100 Hz) and number of channels (mono, 1 channel, or stereo, 2 channels). The higher the sampling rate the higher the audio frequency that can be received, but the more work your software needs to do. Normally you should choose the lowest sampling rate suitable for your application, remembering that it needs to be at least **double** the highest audio frequency in which you are interested (according to the [[http://en.wikipedia.org/wiki/Nyquist_rate|Nyquist criterion]]).\\ \\  You set up the required audio format and open the wave input device as follows:
+<code bb4w>
         DIM Format{wFormatTag{l&,h&}, nChannels{l&,h&}, nSamplesPerSec%, \
         \          nAvgBytesPerSec%, nBlockAlign{l&,h&}, wBitsPerSample{l&,h&}, \
@@ Line 23: / Line 183: @@
         SYS "waveInOpen", ^WaveIn%, _WAVE_MAPPER, Format{}, 0, 0, 0 TO ret%
         IF ret% ERROR 100, "waveInOpen failed: "+STR$~ret%
-In this example a sampling rate of 44100 Hz has been selected. Note that you cannot be sure that the incoming audio is actually being sampled at the specified rate. On some PCs the sampling rate may be as low as 11025 Hz even if a higher frequency has been selected in your program.\\ \\
+</code>
+In this example a sampling rate of 44100 Hz has been selected. Note that you cannot be sure that the incoming audio is actually being sampled at the specified rate. On some PCs the sampling rate may be as low as 11025 Hz even if a higher frequency has been selected in your program.
 ==== Creating and initialising the buffers ====
-\\  The next step is to decide how many audio buffers you need and how large they should be. To some extent this is an arbitrary decision, but it will depend on things like //latency// (how much time elapses between the audio arriving and it being processed by your program) and the amount of work needed to process the received audio data.\\ \\  Normally you should have at least three buffers: one inputting the sampled sound, one being processed by your program, and one spare (the buffers are reused cyclically). It is vitally important that your program can process the audio data quickly enough, otherwise data will be lost, with undesirable results. If there is any variability in the rate at which you can process the data (for example it depends on disk or network accesses) then you may need to use more and/or larger buffers to 'iron out' the fluctuation. Using more buffers is generally preferable to using larger buffers, to minimise any increase in latency.\\ \\  In the example below the number of buffers is three and the length of each buffer is 1024 samples; at 44100 Hz that implies a latency of at least 24 milliseconds. The code for creating and initialising the buffers is as follows:\\ \\
+The next step is to decide how many audio buffers you need and how large they should be. To some extent this is an arbitrary decision, but it will depend on things like //latency// (how much time elapses between the audio arriving and it being processed by your program) and the amount of work needed to process the received audio data.\\ \\  Normally you should have at least three buffers: one inputting the sampled sound, one being processed by your program, and one spare (the buffers are reused cyclically). It is vitally important that your program can process the audio data quickly enough, otherwise data will be lost, with undesirable results. If there is any variability in the rate at which you can process the data (for example it depends on disk or network accesses) then you may need to use more and/or larger buffers to 'iron out' the fluctuation. Using more buffers is generally preferable to using larger buffers, to minimise any increase in latency.\\ \\  In the example below the number of buffers is three and the length of each buffer is 1024 samples; at 44100 Hz that implies a latency of at least 24 milliseconds. The code for creating and initialising the buffers is as follows:
+<code bb4w>
         nBuffers% = 3
         SamplesPerBuffer% = 1024
@@ Line 47: / Line 213: @@
           IF ret% ERROR 100, "waveInAddBuffer failed: "+STR$~ret%
         NEXT
-Note that in this case the audio buffers are allocated from BASIC's heap; you could alternatively use the Windows API to allocate the memory.\\ \\
+</code>
+Note that in this case the audio buffers are allocated from BASIC's heap; you could alternatively use the Windows API to allocate the memory.
 ==== Starting audio capture ====
-\\  Once you have prepared the wave input system using the above code, you can start the real-time capture as follows:\\ \\
+Once you have prepared the wave input system using the above code, you can start the real-time capture as follows:
+<code bb4w>
         SYS "waveInStart", WaveIn% TO ret%
         IF ret% ERROR 100, "waveInStart failed: "+STR$~ret%
-\\
+</code>
 ==== Inputting in real-time ====
-\\  Once the above code has been executed you need to process the received audio buffers fast enough to keep up with the incoming data. The following code constantly checks whether any of the buffers needs processing and if so calls the **PROCprocessbuffer** routine:\\ \\
+Once the above code has been executed you need to process the received audio buffers fast enough to keep up with the incoming data. The following code constantly checks whether any of the buffers needs processing and if so calls the **PROCprocessbuffer** routine:
+<code bb4w>
         _WHDR_DONE = 1
         REPEAT
@@ Line 66: / Line 242: @@
           SYS "Sleep", 1
         UNTIL FALSE
-In this example the audio processing continues indefinitely, but you can terminate the program prematurely if you wish. If you do, don't forget to execute **PROCcleanup** before exiting the program.\\ \\
+</code>
+In this example the audio processing continues indefinitely, but you can terminate the program prematurely if you wish. If you do, don't forget to execute **PROCcleanup** before exiting the program.
 ==== Processing the audio data ====
-\\  Obviously it's only possible to describe this aspect in general terms, because precisely what audio processing takes place will depend on what the program is designed to do. The code below simply calculates the RMS (Root Mean Square) value of the incoming audio:\\ \\
+Obviously it's only possible to describe this aspect in general terms, because precisely what audio processing takes place will depend on what the program is designed to do. The code below simply calculates the RMS (Root Mean Square) value of the incoming audio:
+<code bb4w>
         DEF PROCprocessbuffer(B%,N%)
         LOCAL I%, V%, sumsq
@@ Line 77: / Line 259: @@
         RMS = SQR(sumsq / N%)
         ENDPROC
-This code is appropriate for monaural input (one channel) where each audio sample consists of a signed 16-bit value in the range -32768 to +32767.\\ \\
+</code>
+This code is appropriate for monaural input (one channel) where each audio sample consists of a signed 16-bit value in the range -32768 to +32767.
 ==== Cleaning up ====
-\\  When you stop the sound capture, or exit the program, you need to shut down the audio input in a controlled fashion:\\ \\
+When you stop the sound capture, or exit the program, you need to shut down the audio input in a controlled fashion:
+<code bb4w>
         DEF PROCcleanup
         WaveIn% += 0 : IF WaveIn% THEN
@@ Line 88: / Line 276: @@
         ENDIF
         ENDPROC
+</code>
 This code might form part of a larger routine, if there are other things that need to be shut down.