邢台网站建设多少钱,做兼职打字员的网站,网站建设微信官网开发,专门做mmd的网站录制程序要添加新功能#xff1a;录制CMMB电视节目#xff0c;我们的板卡发送出来的是RTP流(H264视频和AAC音频)#xff0c;录制程序要做的工作是#xff1a;(1)接收并解析RTP包#xff0c;分离出H264和AAC数据流#xff1b;(2)将H264视频和AAC音频以某种格式封装#x…录制程序要添加新功能录制CMMB电视节目我们的板卡发送出来的是RTP流(H264视频和AAC音频)录制程序要做的工作是(1)接收并解析RTP包分离出H264和AAC数据流(2)将H264视频和AAC音频以某种格式封装最后存成文件供用户查看。第一步已经有部分代码可供参考因此很快就完成了。第二步我们决定封装成mp4查找了一些资料后决定使用开源库mp4v2来合成mp4文件。技术路线已确定就开工干活。(一)mp4格式的基础知识。关于mp4格式网上介绍的不少有以下内容可供参考(1)两个ISO标准[ISO/IEC 14496-12]:ISO base media file format --”is a general format forming the basis for a number of other more specific file formats. This format contains the timing, structure, and media information for timed sequences of media data, such as audio-visual presentations ”[ISO/IEC 14496-14]:MP4 file format --”This specification defines MP4 as an instance of the ISO Media File format [ISO/IEC 14496-12 and ISO/IEC15444-12]. ”定义了mp4文件格式标准。是上面两个标准的解释建议先看这个了解大概具体细节再看ISO标准文件。(二)技术验证。主要就是写验证代码验证技术可行性。去官网下载mp4v2源码、编译、安装过程略过不提。所有资料可以在http://code.google.com/p/mp4v2/找到。先写部分验证代码很快完成了但封装出来的文件有问题无法播放。合成部分代码如下static void* writeThread(void* arg){rtp_s* p_rtp (rtp_s*) arg;if (p_rtp NULL){printf(ERROR!\n);return;}MP4FileHandle file MP4CreateEx(test.mp4, MP4_DETAILS_ALL, 0, 1, 1, 0, 0, 0, 0);if (file MP4_INVALID_FILE_HANDLE){printf(open file fialed.\n);return;}MP4SetTimeScale(file, 90000);//添加h264 trackMP4TrackId video MP4AddH264VideoTrack(file, 90000, 90000 / 25, 320, 240,0x64, //sps[1] AVCProfileIndication0x00, //sps[2] profile_compat0x1f, //sps[3] AVCLevelIndication3); // 4 bytes length before each NAL unitif (video MP4_INVALID_TRACK_ID){printf(add video track failed.\n);return;}MP4SetVideoProfileLevel(file, 0x7F);//添加aac音频MP4TrackId audio MP4AddAudioTrack(file, 48000, 1024, MP4_MPEG4_AUDIO_TYPE);if (video MP4_INVALID_TRACK_ID){printf(add audio track failed.\n);return;}MP4SetAudioProfileLevel(file, 0x2);int ncount 0;while (1){frame_t* pf NULL; //framepthread_mutex_lock(p_rtp-mutex);pf p_rtp-p_frame_header;if (pf ! NULL){if (pf-i_type 1)//video{MP4WriteSample(file, video, pf-p_frame, pf-i_frame_size, MP4_INVALID_DURATION, 0, 1);}else if (pf-i_type 2)//audio{MP4WriteSample(file, audio, pf-p_frame, pf-i_frame_size , MP4_INVALID_DURATION, 0, 1);}ncount;//clear frame.p_rtp-i_buf_num--;p_rtp-p_frame_header pf-p_next;if (p_rtp-i_buf_num 0){p_rtp-p_frame_buf p_rtp-p_frame_header;}free_frame(pf);pf NULL;if (ncount 1000){break;}}else{//printf(BUFF EMPTY, p_rtp-i_buf_num:%d\n, p_rtp-i_buf_num);}pthread_mutex_unlock(p_rtp-mutex);usleep(10000);}MP4Close(file);}现象没有图像也没有声音根本无法播放。于是艰苦的工作开始了跟踪查找原因。(1)使用 vlc播放合成的mp4文件查看详细输出vlc -vvv test.mp4[0x8e9357c] mp4 stream debug: found Box: ftyp size 24[0x8e9357c] mp4 stream debug: found Box: free size 136[0x8e9357c] mp4 stream debug: skip box: free[0x8e9357c] mp4 stream debug: found Box: mdat size 985725[0x8e9357c] mp4 stream debug: skip box: mdat[0x8e9357c] mp4 stream debug: found Box: moov size 5187[0x8e9357c] mp4 stream debug: found Box: mvhd size 108[0x8e9357c] mp4 stream debug: read box: mvhd creation734515d-06h:22m:03s modification 734515d-06h:22m:23stime scale 90000 duration 694977d-48h:00m:29srate 1.000000 volume 1.000000 next track id 3可以看到vlc(实际上是调用libmp4库)解析box都正确的mdat的大小也是正确的。但接下来一行skip box: mdat就比较奇怪了明明解析正确了为什么要将mdat忽略掉呢要知道mdat里存放的可是真正的音视频数据阿如果skip掉了后面解码时没有数据当然播放不了了(2)既然找到疑点继续跟踪。查看vlc的源代码在文件modules/demux/mp4/libmp4.c中发现skip信息是由MP4_ReadBoxSkip()函数打印的而调用的地方在libmp4.c中2641行/* Nothing to do with this box */{ FOURCC_mdat, MP4_ReadBoxSkip, MP4_FreeBox_Common },{ FOURCC_skip, MP4_ReadBoxSkip, MP4_FreeBox_Common },{ FOURCC_free, MP4_ReadBoxSkip, MP4_FreeBox_Common },{ FOURCC_wide, MP4_ReadBoxSkip, MP4_FreeBox_Common },而在libmp4.h中#define FOURCC_mdat VLC_FOURCC( m, d, a, t )#define FOURCC_skip VLC_FOURCC( s, k, i, p )#define FOURCC_free VLC_FOURCC( f, r, e, e )#define FOURCC_wide VLC_FOURCC( w, i, d, e )从代码看vlc调用libmp4解析文件时主动忽略了mdat,skip,free,wide这四种类型的box。为什么呢(3)继续查看modules/demux/mp4/mp4.c中的Open()函数(解析模块的入口函数)可以看到本模块的主要工作是初始化一个demux_sys_t结构体该结构体定义如下struct demux_sys_t{MP4_Box_t *p_root; /* container for the whole file */mtime_t i_pcr;uint64_t i_time; /*time position of the presentation * in movie timescale*/uint64_t i_timescale; /* movie time scale */uint64_t i_duration; /* movie duration */unsigned int i_tracks; /* number of tracks */mp4_track_t *track;/* array of track */float f_fps; /* number of frame per seconds *//* */MP4_Box_t *p_tref_chap;/* */input_title_t *p_title;};似乎只是为了获取mp4的tracks,moov,duration, timescale等基本信息实际上并不解码数据因此就不需要关注mdat这个box了。综上vlc的输出是正常的libmp4忽略了mdat这个box也不是造成mp4文件无法播放的原因只是因为libmp4这个模块并不真正解码数据所以不需要关注这个box。既然问题不在这那在哪里呢(4)继续看vlc的输出AVC: nal size -1710483062no frame![0x8e93eb4] avcodec decoder warning: cannot decode one frame (3595 bytes)可以看到vlc实际上是调用avcodec(ffmpeg)来解码数据的我们的视频是AVC(H264)格式的。从错误信息可以确定是H264的NAL大小错误似乎跟mp4文件本身关系不大。不管那么多先看看代码再说。vlc是以lib的形式使用ffmpeg的所以我们必须看ffmpeg的代码libavcodec/h264.c:static int decode_nal_units(H264Context *h, const uint8_t *buf, int buf_size){….for(;;){if(buf_index next_avc) {if(buf_index buf_size) break;nalsize 0;for(i 0; i nal_length_size; i)nalsize (nalsize if(nalsize 0 || nalsize buf_size - buf_index){av_log(h-s.avctx, AV_LOG_ERROR, AVC: nal size %d\n, nalsize);break;}next_avc buf_index nalsize;}…}可以看到正是这里报错的。但是为什么报错呢根据ffmpeg的信息知道取出来的 nalsize为负数。怀疑是h264流本身有问题于是用Elecard查看了生成的mp4文件视频播放又非常正常。似乎h264流是正常的愁呀愁。。。。内容如下Ottavio Campana“question about MP4AddH264VideoTrack。Whats the meaning of the profile_compat andsampleLenFieldSizeMinusOne fields?”Jeremy NoringUsually an NALU is prefixed by the start code 0x00000001. To write itas a sample in MP4 file format, just replace the start code with sizeof the NALU(without 4-byte start code) in big endian. You also need tospecify how many bytes of the size value requires. Take libmp4v2 forexample, the last parameter in MP4AddH264VideoTrack(.., uint8_tsampleLenFieldSizeMinusOne) indicate the number of byes minus one....so each sample you and to mp4v2 should be prefixed with a size code(in big-endian, of course). I use a 4 byte size code, sosampleLenFieldSizeMinusOne gets set to 3. This seems to work; myfiles playback on just about everything. Perhaps one of the projectmaintainers can clarify this, and itd also be good to update thedocumentation of that call to make this clear.”Ottavio Campanathats the code I used as reference to write my program :-(but my doubt is that there must be something wrong somewhere, becauseboxes seem to be correctly written, but when I try to decode them Iget errors like[h264 0xb40fa0]AVC: nal size -502662121have you ever seen an error like this?Ottavio Campana Not sure, but it looks youre not converting it to big-endian before prefixing it to your sample.well, eventually using ffmpeg to dump the read frames, I discoveredthat I had to strip che NALU start code, i.e. the 0x00000001, and toput the NALU size at its place.It works perfectly now, but I still wonder why I had to put the sizeat the begin of the data, since it is a parameter which is passed toMP4WriteSample, so I expected the function to add it.从中得到如下关键信息(1)h264流中的NAL头四个字节是0x00000001;(2)mp4中的h264track头四个字节要求是NAL的长度并且是大端顺序(3)mp4v2很可能针对此种情况并没有做处理所以写到mp4文件中的每个NAL头四个字节还是0x00000001.那好说我将每个sample(也就是NAL)的头四个字节内容改成NAL的长度且试试看if(pf-i_frame_size 4){uint32_t* p (pf-p_frame[0]);*p htonl(pf-i_frame_size -4);//大端,去掉头部四个字节}MP4WriteSample(file, video, pf-p_frame, pf-i_frame_size, MP4_INVALID_DURATION, 0, 1);测试下来果然OK了(6)视频已经解决了音频还有问题播放的声音太快。尝试调整参数MP4TrackId audio MP4AddAudioTrack(file, 48000, 1024, MP4_MPEG4_AUDIO_TYPE);第三个参数sampleDuration表示每个sample持续多少个duration网上看到的都是1024。我尝试了几个不同的值128,256,512,4096都不行最后发现设为2048就正常了。(为什么是2048??????我不清楚也许是因为我们的音频是双声道有时间再研究。。。)正确代码如下MP4TrackId audio MP4AddAudioTrack(file, 48000, 2048, MP4_MPEG4_AUDIO_TYPE);至此已经成功的将rtp流合成了mp4文件证明了技术上是可行的。注意很多参数都是针对我们的具体应用写死的仅供参考。(三)将功能合并到录制程序中。略。from:http://www.rosoo.net/a/201305/16631.html
