Implementing system sound recording in Android-RK3399 development board source code modification

Preface

I have been recording system sound these days and succeeded after modifying the source code and compiling it. Now I will introduce the recording solution of the built-in sound source in detail.

In Android, you can use MediaRecorder.AudioSource.REMOTE_SUBMIX to record system sounds. This property is only available to system applications.

Moreover, this property will cut off the sound from the earphones and speakers, so you won't be able to hear the sound when music or videos are playing on the phone.

At this time we have to change the system source code.

1. Simple Implementation of Recording

 AudioRecord audioRecord = new AudioRecord ( MediaRecorder . AudioSource . REMOTE_SUBMIX
 , captureSampleRate
 , captureChannel
 , AudioFormat .ENCODING_PCM_16BIT
 , recordBufferSize );
 new Thread ( new Runnable () {
 @Override
 public void run () {
 final byte [] bytes = new byte [ recordBufferSize ];
 audioRecord . read ( bytes , 0 , bytes . length );
 audioRecord .setPositionNotificationPeriod ( captureSampleRate / 25 ) ;
 audioRecord . setRecordPositionUpdateListener ( new AudioRecord . OnRecordPositionUpdateListener () {
 @Override
 public void onMarkerReached ( AudioRecord recorder ) {
 }
 @Override
 public void onPeriodicNotification ( AudioRecord recorder ) {
 singleThreadPool . execute ( new Runnable () {
 @Override
 public void run () {
 try {
 if ( audioRecord != null ) {
 audioRecord . read ( bytes , 0 , bytes . length );
 PcmBuffer.clear () ;
 PcmBuffer . put ( bytes , 0 , recordBufferSize );
 audioFrameParam .sampleRate = ZEGO_AUDIO_SAMPLE_RATE_44K ;
 //Sound processing
 }
 } catch ( Exception e ) {
 e . printStackTrace ();
 }
 }
 });
 }
 });
 audioRecord .startRecording ();
 }
 }). start ();

2. Introduction to AudioSource input source

 public final class AudioSource {
 private AudioSource () {}
 /** Default audio source **/
 public static final int DEFAULT = 0 ;
 /** Microphone audio source */
 public static final int MIC = 1 ;
 /** Voice call uplink (Tx) audio source */
 public static final int VOICE_UPLINK = 2 ;
 /** Voice call downlink (Rx) audio source */
 public static final int VOICE_DOWNLINK = 3 ;
 /** Voice call uplink + downlink audio source */
 public static final int VOICE_CALL = 4 ;
 /** Microphone audio source with same orientation as camera if available, the main
 * device microphone otherwise */
 public static final int CAMCORDER = 5 ;
 public static final int VOICE_RECOGNITION = 6 ;
 public static final int VOICE_COMMUNICATION = 7 ;
 public static final int REMOTE_SUBMIX = 8 ;
 }

• DEFAULT: Defaults to MIC, android.permission.RECORD_AUDIO.
• MIC: Microphone, android.permission.RECORD_AUDIO.
• VOICE_UPLINK: Phone recording uplink line, android.permission.CAPTURE_AUDIO_OUTPUT. System permissions do not allow third-party apps to use this.
• VOICE_DOWNLINK: Phone recording downlink line, android.permission.CAPTURE_AUDIO_OUTPUT. System permissions do not allow third-party apps to use this.
• VOICE_CALL: Call recording, android.permission.CAPTURE_AUDIO_OUTPUT. System permissions do not allow third-party apps to use this.
• CAMCORDER: Camera microphone, android.permission.RECORD_AUDIO.
• VOICE_RECOGNITION: voice recognition, android.permission.RECORD_AUDIO.
• VOICE_COMMUNICATION: Internet phone call, android.permission.RECORD_AUDIO.
• REMOTE_SUBMIX: The audio mixing stream transmitted to the remote. By default, if you use this item to record, the sound of the local speaker or headphones will be intercepted. android.permission.CAPTURE_AUDIO_OUTPUT, the system permission does not allow third-party apps to use it.

Notes on using REMOTE_SUBMIX

(1) System permissions are required.

(2) The sound from the speakers and headphones will be intercepted, which means that the sound cannot be played locally when recording.

For system permissions, you need to add android:sharedUserId="android.uid.system" in AndroidManifest.xml, and then use the system signature to package the application. In this way, the third-party application is packaged as a system application and can use system permissions.

3. Source code modification

1. Screen recording on Android 10 and below

 frameworks \av \services \audiopolicy \enginedefault \src \Engine . cpp
 if ( mAvailableOutputDevices . getDevice ( AUDIO_DEVICE_OUT_REMOTE_SUBMIX , ​​String8 ( "0" )) != 0 ) {
 device2 = availableOutputDeviceTypes & AUDIO_DEVICE_OUT_REMOTE_SUBMIX ;
 }

Modified to:

 if ( mAvailableOutputDevices . getDevice ( AUDIO_DEVICE_OUT_REMOTE_SUBMIX , ​​String8 ( "0" )) != 0 ) {
 device2 = availableOutputDeviceTypes & AUDIO_DEVICE_OUT_REMOTE_SUBMIX ;
 device2 |= ( availableOutputDeviceTypes & AUDIO_DEVICE_OUT_WIRED_HEADPHONE );
 device2 |= ( availableOutputDeviceTypes & AUDIO_DEVICE_OUT_SPEAKER );
 }

2. Android 11

 targetframeworks \av \services \audiopolicy \enginedefault \src \Engine . cpp
 if (( remoteSubmix = availableOutputDevices . getDevice (
 AUDIO_DEVICE_OUT_REMOTE_SUBMIX , ​​String8 ( "0" ),
 AUDIO_FORMAT_DEFAULT )) != nullptr ) {
 devices2.add ( remoteSubmix ) ;
 }

Modified to:

 if (( remoteSubmix = availableOutputDevices . getDevice (
 AUDIO_DEVICE_OUT_REMOTE_SUBMIX , ​​String8 ( "0" ),
 AUDIO_FORMAT_DEFAULT )) != nullptr ) {
 devices2 = availableOutputDevices . getDevicesFromTypes ({
 AUDIO_DEVICE_OUT_REMOTE_SUBMIX , ​​AUDIO_DEVICE_OUT_WIRED_HEADPHONE , AUDIO_DEVICE_OUT_SPEAKER });
 }

Conclusion

To record system sound on Android, it is not available under normal circumstances. This solution is to modify the source code to achieve it.

Later I can introduce how the system is implemented, and post the source code so that we can learn together.