Opened 18 years ago
Closed 17 years ago
#1035 closed task (fixed)
EIT rewrite
Reported by: | Stuart Auchterlonie | Owned by: | danielk |
---|---|---|---|
Priority: | minor | Milestone: | 0.21 |
Component: | eit | Version: | head |
Severity: | medium | Keywords: | |
Cc: | Ticket locked: | no |
Description
This ISN'T a wishlist. This is what we need to do to rewrite the EIT handling for 0.20
#778 - Astra 19.2E EPG aka. remove Kristian Kalweit fix
#892 - Handling channels with high EIT data rates.
#913 - Handling of EIT data with additional data
#1025 - Handling of unicode characters within EIT data.
Split all eitFixup routines out of siparser. Use them on EIT data only after we have decided that this is new/updated information.
Keep track of which events we have sent to the database. ie. do our own caching of EIT data, getting rid of that currently done for events in siparser (also #943) (GreyFoxx has done most of this already)
EIT data prioritization. which is better EIT or xmltv? Within EIT, lower table numbers always seem to be better data. 0x4e & 0x4f now & next, 0x50 & 0x51 future, 0x60 & 0x61 far future, and other longer term EIT data from various providers
Fixup routines sometimes try and put a subtitle in the DB that exceeds the length of the subtitle field in the DB, so silent truncation of information occurs.
Attachments (7)
Change History (66)
comment:1 Changed 18 years ago by
Owner: | changed from danielk, stuarta to anonymous |
---|---|
Status: | new → assigned |
comment:2 Changed 18 years ago by
Owner: | changed from anonymous to danielk, stuarta |
---|---|
Status: | assigned → new |
comment:3 Changed 18 years ago by
comment:4 Changed 18 years ago by
Owner: | changed from danielk, stuarta to danielk |
---|
Reassigning, so this shows up on my ticket list
comment:5 Changed 18 years ago by
comment:7 Changed 18 years ago by
comment:8 Changed 18 years ago by
comment:9 Changed 18 years ago by
comment:10 Changed 18 years ago by
comment:12 Changed 18 years ago by
comment:13 Changed 18 years ago by
comment:14 Changed 18 years ago by
comment:15 Changed 18 years ago by
comment:16 Changed 18 years ago by
comment:17 Changed 18 years ago by
(In [9118]) Fixes #1375, References #1035. Allow each A/V stream to use multiple Conditional Access methods.
Apparently this is used in Australia. I'm not sure if they use different methods for the video, audio, and data streams, or if they just list all the supported methods and hope one is sticks. Multiple CA methods per PMT was already supported.
comment:18 Changed 18 years ago by
comment:19 Changed 18 years ago by
comment:20 Changed 18 years ago by
comment:21 Changed 18 years ago by
comment:22 Changed 18 years ago by
comment:23 Changed 18 years ago by
comment:24 Changed 18 years ago by
Hi,
I am getting a crash on startup with 9664-8. I've tried 9663 which is fine. Looking at the GDB output, it appears to be a problem in the EIT scanner.
I've attached some backtraces from 9666 and 9666.
HTH
Chris
Changed 18 years ago by
Attachment: | 9664-gdb.txt added |
---|
Changed 18 years ago by
Attachment: | 9666-gdb.txt added |
---|
Changed 18 years ago by
Attachment: | 9666-myth.log added |
---|
Changed 18 years ago by
Attachment: | 9664-myth.log added |
---|
comment:25 Changed 18 years ago by
(In [9669]) References #1035. This moves most of the EIT parsing for DVB out of SIParser and into the eit classes.
I haven't ported the fixup functions over yet. The new structure depends on being able to de-link eit event processing from the channel/transport we are currently on in the SIParser. But a lot of the "PrivateType?" stuff for the fixups is not cached.
comment:27 Changed 18 years ago by
comment:28 Changed 18 years ago by
comment:29 Changed 18 years ago by
comment:30 Changed 18 years ago by
comment:31 Changed 18 years ago by
comment:32 Changed 18 years ago by
comment:33 Changed 18 years ago by
(In [9743]) References #1035, References #1701.
You may need to do a distclean. This removes signals and slots from the channels and recorders.
I've tested this with every recorder I have access to, but I don't have access to a firewire or dbox2 recorder.
If you experience any problems please do a distclean before reporting the problem.
The reasons for getting rid of signals and slots in non gui elements are because they require delayed deletes and other extra complexity to avoid segfaults, and because they are not thread safe and are incompatible with virtual inheritence and multiple inheritence.
comment:34 Changed 18 years ago by
comment:35 Changed 18 years ago by
comment:36 Changed 18 years ago by
(In [9798]) Refs #1035. Refs #1049. This is a continuation of the refactor started in [9790].
This makes it possible for DVBRecorder and SIParser to use a DVBStreamData in addition to a ATSCStreamData.
This allows us to move more of the DVB specific EIT stuff from SIParser to DVBStreamData.
I also referenced #1049, because this lets us use a DVBStreamData in DVBSignalMonitor and then pass it along to both the DVBRecorder and DVBSIParser; which simplifies the implementation of NIT/SDT monitoring.
This also adds SetStreamData?() and GetStreamData?() stubs to DTVRecorder, this allows us to get rid of a few of the ifdef's in TVRec. TVRec has also been modified to use a DVBStreamData rather than an ATSCStreamData when starting a signal monitor for a non-ATSC digital channel.
comment:37 Changed 18 years ago by
comment:38 Changed 18 years ago by
(In [9816]) Refs #1035. More EIT refactoring for EIT branch.
The major change is the removal of the DVBSIParser and the code it depends on. DTVSignalMonitor picks up it's small remaining role in active EIT scanning. This in turn enables EIT scanning for all recorders with a DTVSignalMonitor based signal monitor.
This patch looks a lot bigger than it is. Many of the changes are deletions of files we no longer use now that [DVB]SIParser is gone.
By replacing the dvbdevice() function, which was our only remaining dependence on dvbdev aside from dvbci, we can also handle more than 4 DVB devices now. Admitedly this is not a common problem, but there is no good reason for this limitation.
comment:39 Changed 18 years ago by
comment:40 Changed 18 years ago by
(In [9831]) Refs #1035. Refactors DVB recording a bit to work better with StreamData? classes (which is being used for more generalized EIT handling).
In order allow the recorder to monitor the PAT & PMT for the stream this makes DVBRecorder and not get confused by multiple PMTs on a pid this moves the HandlePAT() and HandlePMT() methods from the DVBChannel to the DVBRecorder.
This moves some the channel table DB stuff from TVRec to ChannelUtil? as a cleanup. This also to allows us to more easilly check whether two channels share a multiplex by untie these queries from the current channel in the channel class.
This also adds the other pids in the PAT to the pids being monitored in DVBRecorder, this allows us to more quickly change between channels on the same transport by simply querying if we have the needed PSIP tables and using them if they are present.
So far this has been tested with the HDHomeRun and DVB recorders, but it should also speed up FireWire? and HDTV recorder channel changing since the use the StreamData? classes.
comment:41 Changed 18 years ago by
comment:42 Changed 18 years ago by
(In [9844]) Refs #1035. Fixes #1049. Adds SDT monitoring to DTVSignalMonitor in mythtv-eit branch.
What this does is require a matching SDT table for DVB channels before decide that we are on the right transport. This especially helps with DVB-S tuning, but also lets us collect the SDT table in the DTVSignalMonitor which allows us to dependably collect DVB EIT now that the SIParser class is gone.
This also changes the scanning for ATSC/MPEG a bit. We've been inserting a networkid of '1' instead of '0' for these transports for a while. There is a db update in dbcheck to fix existing transports and some changes in siscan to insert these properly in future channel scans.
Please test/debug so that I can merge this with SVN-head soon.
comment:43 Changed 18 years ago by
(In [9847]) Refs #1035. merged trunk/mythtv changes r9833:9846 into the mythtv-eit branch.
comment:44 Changed 18 years ago by
(In [9957]) Refs #1035. Big EIT branch merge.
This moves the EIT handling out of SIParser and mostly into the DVB and ATSC stream handlers, allowing recorders other than the DVBRecorder to handle EIT without massive code duplication. This also allows us to get rid of a bunch of dependencies on the dvbdev stuff. The only thing we are still using there is the CAM code.
This also fixes several problems in the MPEG PES packet assembly code that were uncovered when we started using it for EIT in addition to using it for the tuning tables.
Since SIParser is gone this also allows us to use the table caching in the stream handler code to speed up intra-transport tuning. If we already have the needed tuning tables we skip the signal monitoring step and begin recording the new channel immediately.
This also fixes various bugs in EIT handling, and lowers the EIT processors memory consumtion further.
This also removes the arbitrary limit of four DVB cards per backend. Now you should be able to add as many DVB devices as your hardware allows.
I recommend a "make distclean".
comment:45 Changed 18 years ago by
comment:46 Changed 18 years ago by
comment:47 Changed 18 years ago by
(In [10238]) Refs #1035. Refs #1728. Refs #1801. Refs #1884. Refs #1615. EIT branch merge.
- Improves the EIT Fixup functions for the UK and Sweden.
- Improves the EIT version cache (lowering memory and CPU requirements).
- Fixes various small memory leaks and lowers memory requirements with earlier deletes.
- Fixes a spurious error message in filldata with EIT only sources.
comment:48 Changed 18 years ago by
(In [10244]) Refs #1035. Undo overly Janne's leak fix for StreamData? in tv_rec. It improperly deletes stream data in some cases.
comment:49 Changed 18 years ago by
Milestone: | 0.20 → 0.21 |
---|
comment:50 Changed 18 years ago by
Priority: | major → minor |
---|
comment:52 Changed 18 years ago by
comment:53 Changed 17 years ago by
(In [11752]) Refs #1035. Several European DVB providers don't set the character set for their EIT listings as required by the DVB standards so we have been applying an override to set it to a sensible value; but, for some reason the 'sensible value' we chose was the Turkish character set. In all likelyhood they are using something closer to the Western European character set, so I've changed the character set appropriately.
The two character sets are very similar, they differ mostly in how they handle some special characters which is why this wasn't caught earlier.
Changed 17 years ago by
Attachment: | malta_dtv_multiplex_dump added |
---|
DTV_mulitplex dump as requested by Daniel
comment:54 Changed 17 years ago by
Change set 11752 introduced an eit bug with Bell Expressvu. The early eit fixups for the character set is overwriting previous defined fixups.
Changed 17 years ago by
Attachment: | early_eit_fixup.diff added |
---|
comment:55 Changed 17 years ago by
comment:56 Changed 17 years ago by
(In [12820]) Refs #2342, #1035. Adds channel looking to eit cache
This should prevent duplicate eit insertions. The channel looks are removed when the master frontend starts or after 24 hours. The eit cache will now written back to the DB after every channel change. I hope this isn't too much stress for anyone's database / backend server. I didn't notice the higher load.
comment:57 Changed 17 years ago by
(In [12831]) Refs #1035. Optimize DVB EIT insertion.
One DVB event information table caries only events of a single channel => look the chanid only once per section up.
Simplifies the eit cache by removing duplicate chanid lookup.
Fixes eit cache issues with the same channel on different sources (reported by Otto Kolsi)
comment:58 Changed 17 years ago by
Component: | mythtv → eit |
---|
comment:59 Changed 17 years ago by
Resolution: | → fixed |
---|---|
Status: | new → closed |
We'll open a new ticket for the next EIT rewrite..
you probably knew this anyway - but just in case it helps with any design decisions:
a) In UK dvb-t the EITFixupStyle2 function can account for ca 40% of the total processing time in LiveTV - its called ~40 times/sec - those Qt regexes really hurt in this function :)
b) For dvb-t I don't believe the scan needs to look at every channel - just every multiplex
I was about to look see if it was practical to skip ParseDVBEIT for say 10% of the packets or even to skip it completely during recording/livetv - ymwv!