Why Am I Getting "unsupported Format" Errors, Reading H.264 Encoded Rtsp Streams With The Android Mediaplayer?

I am trying to show H.264 encoded rtsp video on an Android device. The stream is coming from a Raspberry Pi, using vlc to encode /dev/video1 which is a 'Pi NoIR Camera Board'. vlc-

Solution 1:

After an amazing amount of dead-ends, I can show a H264 RTSP stream on an Android SurfaceView. This answer is only sort of an answer because I still can't address my original three questions, but even full of bug and shortcuts as it is, my 75K apk is a lot better than Vlc for Android or the osmo4 player: It has sub-second latency (at least when the sender and the receiver are on the same wifi router!) and fills the SurfaceView.

A few takeaways, to help anyone trying to do anything similar:

• All input buffers you pass to MediaCodec.queueInputBuffer() must start with the 00 00 01 sync pattern.
• You can configure() and start() the codec right away - but don't queue any 'normal' input buffers until you've see both an SPS (NALU code 7) and PPS (NALU code 8) packet. (These might not be 0x67 and 0x68 - the "nal_ref_idc" bits should be non-zero but will not necessarily be 11. Fwiw, vlc seems to always give me 01.)
• Pass the SPS/PPS packets almost normally - pass the BUFFER_FLAG_CODEC_CONFIG flag to queueInputBuffer(). In particular, don't try to put them in a "csd-0" buffer attached to the MediaFormat!
• When you see (a) missed frame(s) (i.e. you see a jump in RTP sequence number) do not call codec.flush()! Just skip the partial frame, and don't queue up a bufer until the next full frame.