NuPlayer be based on StagefrightPlayer Basic class construction of , Using the underlying ALooper/AHandler Mechanism for asynchronous decoding and playback ,Alooper Round the messages in the queue , Push the message to AHandler Handle , Finally through handleMessage Function callback , Do the corresponding logical processing

NuPlayer The overall structure

MediaPlayerFactory adopt Factory mode establish NuPlayerFactory, And then through NuPlayerFactory establish NuPlayerDriver, Then passed NuPlayerDriver Construct a NuPlayer,NuPlayer As a player , Analysis of design data 、 decode 、 Rendering and other processes

NuPlayer The building process of

Call... At the upper level setDataSource After the function , Begin to arrive at MediaPlayerService Of setDataSource function , adopt getPlayerType Function to get the player type

Player types are enumerated as follows

Then go back to MediaPlayerFactory Look at it getPlayerType function

Next, enter a macro function , The main function of macros is to replace , Improve code execution efficiency , Because it eliminates the need to allocate and release stack frames 、 The ginseng 、 Pass the return value and a series of work

The label of this macro function traverses map Stored in Player factory class , call scoreFactory You can get the player's playback ability , If it is NuPlayer, There is one by default 0.8 Value , Back to ret Namely NuPlayerFactory object , If you get a value of 0.0, To get into getDefaultPlayerType function

The default is NuPlayerFactory

in the light of NuPlayerFactory It will not be created directly NuPlayer, But in NuPlayerDriver Create a NuPlayerDriver

stay NuPlayerDriver Of Constructors in , Created a NuPlayer,NuPlayerDriver It's right NuPlayer Of encapsulation , Inherit MediaPlayerInterface Interface . adopt NuPlayer Can realize the playback function , It is mainly used to notify the above process

NuPlayer Inherited from AHandler, And introduced AMessage, adopt ALooper To process the message , Like its prepareAsync function

It's actually sending a kWhatPrepare news . stay onMessageReceived Function , Receive and process messages ( The function is too long , Extract only kWhatPrepare Partial analysis )

NuPlayer Data analysis module

The main modules of parsing are NuPlayerSource And several classes that inherit it , Such as HTTPLiveSource、RTSPSource、GenericSource etc.

stay NuPlayer Call in setDataSourceAsync After the function

It will be based on Different protocols Choose different Source object , With this Source After the object , send out kWhatSetDataSource news ( The following figure is also a partial screenshot of the function )

Through concrete Source After analyzing the data , And then Source Cast to mSource to Decoder Use , At this time, it contains data related information , One of them HTTPLiveSource, It is mainly used to analyze HLS agreement

When calling HTTPLiveSource Of prepareAsync Function time

It mainly constructs a LiveSession object , adopt LiveSession Inside connectAsync function , Create a session

see onConnect function

Network request , Start getting playlists , Call back after getting the playlist ( The following figure shows the callback function )

The above code is mainly based on URL Back to M3U file , Get the corresponding BandwidthItem, If you are familiar with M3U file , You can know M3U Files have primary and secondary indexes , The following figure shows some examples

Its main function is to Dynamic rate adaptation ,BANDWIDTH The bigger it is , The higher the resolution .BINDWIDTH It's actually bandwidth

If you want to get a Track Information about , Get the video 、 Audio 、 Caption related information , Can pass LiveSession Of getTrackInfo function

This completes the data analysis module , The main function of video source is to get audio and video data information , Finally through mSource Variables are passed to the decoder

NuPlayer Decoding module of

It introduces NuPlayerDecoderBase, This is a base class , The real decoder logic is NuPlayerDecoder.cpp in , It inherits the base class , Let's look at the instantiated decoder first

The above code constructs Decoder object , And it has been carried out initialization and Build decoder

because NuPlayerDecoder Inherited from NuPlayerDecoderBase class , therefore Configure The function will execute NuPlayerDecoder Of Configure function , The final callback onConfigure function , It is located in NuPlayerDecoder.cpp in

from onConfigure The function can see ,NuPlayerDecoder Introduced MediaCodec As a decoder . adopt CreateByType/CreateByComponentName Created Codec object .

NuPlayer The rendering module of

The main function

1. Cache the original audio and video data to the queue
2. Audio data consumption playback
3. Video data consumption display
4. Audio video synchronization
5. Player control

Cache the original audio and video data to the queue , stay NuPlayerRenderer.h in , There is one. QueueEntry Structure and two queues

The data is in onQueueBuffer Function to add

And the playback of audio is NuPlayerRenderer in , Will call first openAudioSink function

Send a kWhatOpenAudioSink news , After receiving the message, carry out the following processing

call onOpenAudioSink function

It's open here AudioSink( Audio back end ). When putting audio data into the audio queue , Will then call postDrainAudioQueue_l function

The code above sends a kWhatDrainAudioQueue

There is mainly one to judge onDrainAudioQueue function , Decide whether you need to go back to AudioSink Write data

Come here , The audio playback process is as follows ： Open the audio back end first , Then when sending data to the audio queue , The audio queue writes data to the audio backend at the same time , For audio playback

If the video shows , Also after the original video data enters the video queue , Start execution postDrainVideoQueue function

Here, first send a kWhatDrainVideoQueue The news of , Then synchronize audio and video , After sending the message, it will start to call onDrainVideoQueue

NuPlayer::Renderer It uses Based on video The synchronization mechanism of , If the decoded audio data timestamp is greater than the video data timestamp , Discard audio packets directly , Then render the video directly . The synchronization mechanism is mainly located in the video buffer processing part onDrainVideoQueue And audio buffer processing onDrainVideoQueue in . The rendering of audio and video adopts A timer like mechanism , But the video display needs to rely on Actual decoder , Audio playback depends on AudioSink Interface