Opened 15 years ago
Closed 15 years ago
Last modified 15 years ago
#5971 closed defect (fixed)
stream reading eats up all memory in all myth apps
Reported by: | Owned by: | Janne Grunau | |
---|---|---|---|
Priority: | major | Milestone: | 0.22 |
Component: | Video Playback | Version: | head |
Severity: | medium | Keywords: | libmythtv libavformat memory AVFormatDecoder |
Cc: | Ticket locked: | no |
Description
Recently, I noticed a server halting. mythbackend on preview generation took up all memory. Upon further inspection, both mythtranscode and mythcommflag do the same.
mythcommflag on a 20MB sample stream eats up 120MB of memory - for full recordings, the system runs OOM. Of course, I tried to "rebuild" seektables, but the bug seems to be "before" that.
From valgrind's/massif's output, it seems that from AVFormatDecoder::OpenFile on, av_malloc'ed and av_mallocz'ed memory is never freed.
Attachments (4)
Change History (17)
Changed 15 years ago by
Attachment: | valgrind.two added |
---|
comment:1 Changed 15 years ago by
Unfortunately, I don't see through the whole allocing/freeing businness in libavformat.
I found, however, that removing a special "stream" from the mpeg ts changes the memory behaviour:
I have the following streams in the file: 0xA0 => MPEG2VIDEO 0xA1 => MP2 0xA3 => AC3 0xA5 => Teletext (TTX) 0xA9 => VPS
It seems, myth recognizes 0xA9 as "second" video stream, which it isn't (http://en.wikipedia.org/wiki/Video_Program_System). Removing 0xa9 from the file (mpegdemux -r / project-x) is a work-around for me.
As VPS is not interpreted by mythtv anyway, and there is no information in VPS to keep, such streams may be safely discarded.
Changed 15 years ago by
Attachment: | massif.new.ms added |
---|
with eliminated VPS stream; massif log file (ms_printed)
comment:2 Changed 15 years ago by
If I eliminate the VPS stream but keep the TS format, everything works fine (see massif.new log)
comment:3 Changed 15 years ago by
Correction... it seems that all recordings from ORF (Austria) are somehow buggy. I don't know what, the usual tools don't display anything unusual, but a plain conversion through project-x (with all streams intact) solves the problem, too.
Still, bringing down the whole machine is bad(tm). AVFormatDecoder should probably just bail out, if something goes awry.
As a workaround:
- userjob to go over all recordings with project-x;
- configure apache to block all preview images (of recordings which might be not yet "cleansed")
comment:4 Changed 15 years ago by
Learning to cope with threaded programs ;) I use mythtranscode --mpeg2 --video --infile to really limit the surroundings.
The first ever call to ac_read_frame_internal at libs/libavformat/utils.c:2162 never returns...
comment:5 Changed 15 years ago by
Just a note to confirm that I've seen this too - on Ubuntu 8.04 64bit - it's weird to see mythbackend eating up to 21GB mem, and also Mac OS/X intel, 10.5.5 with MythFrontend. In both cases, svn 19244.
comment:6 Changed 15 years ago by
I made gdb log all calls to av_malloc and av_free (and av_realloc)... It seems, that after some startup work, some kind of loop allocating 192/64 byte sizes is entered. Those blocks are never freed (at least until the program reaches its ulimit and gets killed).
See gdb.text attached (truncated).
comment:7 Changed 15 years ago by
Working on a less complex sample of 20MB, and with --verbose all, I get the following:
2008-12-07 15:48:41.384 Transcoding from /home/hpoeri/Desktop/trial20.mpg to out20.mpg 2008-12-07 15:48:41.393 Opening /home/hpoeri/Desktop/trial20.mpg 2008-12-07 15:48:41.829 ac-tex damaged at 19 2 2008-12-07 15:48:41.829 Warning MVs not available 2008-12-07 15:48:41.830 concealing 1156 DC, 1156 AC, 1156 MV errors 2008-12-07 15:48:41.857 Could not find codec parameters (Audio: mp3, s16) 2008-12-07 15:48:41.857 Could not find codec parameters (Video: 0x0000) 2008-12-07 15:48:42.233 Input #0, mpegts, from '/home/hpoeri/Desktop/trial20.mpg': 2008-12-07 15:48:42.233 Duration: N/A, bitrate: N/A 2008-12-07 15:48:42.253 Stream #0.0[0x1fa]: Video: mpeg2video, yuv420p, 544x576 [PAR 24:17 DAR 4:3], 3900 kb/s, 25.00 tb(r) 2008-12-07 15:48:42.253 Stream #0.1[0x1fb](ger): Audio: mp3, s16 2008-12-07 15:48:42.253 Stream #0.2[0x1fd](ger): Data: 0x0000 2008-12-07 15:48:42.253 Stream #0.3[0x200]: Video: 0x0000, 90000.00 tb(r) 2008-12-07 15:48:42.254 Skipping invalid audio stream: 1 2008-12-07 15:48:42.254 Skipping unsupported codec 2 on stream 2 2008-12-07 15:48:42.700 Found end of file without finding any frames 2008-12-07 15:48:42.701 Transcoding /home/hpoeri/Desktop/trial20.mpg failed
Which would seem coherent with the no-freeing noticed above. Although the sample file is playable with all relevant players I have, myth's libavformat seems to eat up memory while searching for a frame (or frame boundaries?)
Unfortunately, I don't know anything about mpegts contents, yet.
However, myth really should give up before EOF or OOM ;)
comment:8 Changed 15 years ago by
Owner: | changed from Isaac Richards to janneg |
---|---|
Status: | new → assigned |
This looks like it's closely related to #5016 - This time with an audio channel rather than video.
comment:9 Changed 15 years ago by
Owner: | changed from janneg to Janne Grunau |
---|---|
Status: | assigned → accepted |
comment:10 Changed 15 years ago by
Milestone: | unknown → 0.22 |
---|---|
Priority: | blocker → major |
comment:11 Changed 15 years ago by
Resolution: | → fixed |
---|---|
Status: | accepted → closed |
comment:12 Changed 15 years ago by
comment:13 Changed 15 years ago by
I can confirm that a memory leak matching this still occurs as of svn 19289 - both Mac OS/X & Linux 64bit.
valgrind log file