Friday, September 3rd 2010, 9:13am UTC+2

You are not logged in.

  • Login
  • Register

1

Sunday, December 27th 2009, 11:43pm

ReconstructApSc - Reconstruct missing or corrupt .ap and .sc files of movies

There is a new plugin, ReconstructApSc, which makes it possible to reconstruct missing or corrupt .ap and .sc files of recorded movies. These files are used for precise seeking and winding in the movie, and also for finding the cut positions when using the plugin MovieCut to cut movies. This plugin scans through .ts files, and stores the structural information into the .ap and .sc files.

Examples of situations when this plugin may be useful include:
  • Movies recorded with Enigma2 from before the spring of 2009 doesn't have .sc files, so rewind/fast forward won't work properly.
  • The previous version of MovieCut did not know about .sc, and therefore produced no such files for the cut movies.
  • If you have cut your movies on a PC, or downloaded movies from the web, you may not get any .ap or .sc files.
  • If a specific movie is impossible to seek correctly in, or just gets black or a frozen picture when trying to fast forward or rewind, then one may suspect that the coresponding .ap and .sc have got corrupt for some reason.
  • After a disk crash it may be possible to rescue the .ts files (because their specific structure) but perhaps not the others.
  • Maybe you have removed the .ap and .sc files because they took up disk space and you didn't know what they were used for.
Once installed, the plugin shows up in the menu in the movie list, as "Reconstruct AP/SC". When started, it will give you three alternatives:
  • Reconstruct .ap and .sc for the selected movie.
    If you suspect a specific file has got a corrupt .ap or .sc, this is the choice. Reconstruction will be performed in the background, so you can continue using the dreambox. It will tell you when processing is finished.
  • Reconstruct all missing .ap and .sc files in the directory.
    A typical situation for this choice is that you have a directory with many older movies that don't have any .sc files. Note that if several movies in a directory miss their .sc files, processing will take considerable time and disk activity (count some 5 - 10 minutes per Gigabyte of a movie, depending on the hard disk), since every such movie has to be scanned through. If the process should be interupted for some reason, it will however start over again the next time with the file in which it was interrupted.
  • Check the progress of any existing reconstruction process.
    Especially if you do a whole directory, you may wonder how many files it has processed so far. With this choice you can see the output from the reconstruction process so far, to see which file it is currently processing.
The plugin is available from the Plugins menu (the experimental feed). I hope that it can be of use.

Best Regards
Anders Holst

2

Tuesday, December 29th 2009, 4:57pm

Anders,

First of all: Thanks for the fantastic MovieCut tool. :thumbsup:

I also do have .ts files without the corresponing .ap files, so I was happy when I saw this new tool.

But where exactly can I download it? I can't seem to find the Plugins menu (the experimental feed) :(
Thanks in advance.

vGnp

3

Tuesday, December 29th 2009, 7:08pm

Quoted

But where exactly can I download it?
If you use the "experimental image" (as opposed to the more stable "official release"), then the Plugins menu will automatically give you plugins from the experimental feed. With the Plugins menu I meant:
Main menu -> Plugins -> Download -> Extensions -> reconstructapsc

Alternatively you can fetch it directly from http://dreamboxupdate.com/opendreambox/1…5/experimental/
Scroll down to "enigma2-plugin-extensions-reconstructapsc_experimental", select the version of latest date, and install it manually on the box.

Best Regards
Anders Holst

4

Tuesday, December 29th 2009, 7:17pm

Alternatively you can fetch it directly from http://dreamboxupdate.com/opendreambox/1…5/experimental/
Scroll down to "enigma2-plugin-extensions-reconstructapsc_experimental", select the version of latest date, and install it manually on the box.
Anders,

Thank you very much. I had tried to find it on elitedvb, and Google wasn't really helping also. Again, Thanks.



Best Regards,

vGnp

5

Wednesday, June 16th 2010, 7:00am

Hello,
I'm having problems with this plugin. When I transfer a movie in TS-format from my PC to the DM800 (with[Moderator] Image is in violation of the Boardrules [/Moderator] ) the Movie-Entry displayed by the Mediaplayer contains also a Film-Length (in minutes). This movie can easily be played by the Dreambox.
Now, when I index this movie with the reconstructapsc-plugin, the Film-Length disapears in the Movie-Entry of the Mediaplayer and the movie can no longer be played. It seems that the play-back function ultimately jumps to the end of the video-file. After a remove of *.ts.ap and *.ts.sc one can play the movie again.
Does anyone has an idea what the problem could be?
Thanks, Willi

This post has been edited 1 times, last edit by "floh" (Jun 16th 2010, 10:55am)


6

Wednesday, June 16th 2010, 10:50am

I have the same problems as Willihelmi. Sometimes I transcode HD and HQ files from onlinetvrecorder.com (these are MP4/MP3 in an AVI container) to ts. The .ap and .sc files seems to have wrong timecodes so cutedit/moviecut on the box doesn't work (and I have to do it on the PC in the AVI file first - this isn't as convenient as on the box ;) ).
Anders, if you need a sample file, just send me a "PN".
DM 800 HD PVR DVB-S2 OoZooN OE1.6-2010-07-20, Samsung UE40B6000

7

Friday, July 30th 2010, 3:34pm

I can confirm! The reconstructed .ap and .sc files are messed up and make the movie unplayable.

This post has been edited 1 times, last edit by "DarthMaul40" (Jul 30th 2010, 3:48pm)


8

Sunday, August 1st 2010, 7:58pm

Hey, Anders, nice plugin, but could you please see why it's behaving like this? I am not alone having this problem.

Thanks in advance!

9

Monday, August 2nd 2010, 12:37am

I have tried to look into this, but have not been able to repeat it, mainly because I have no access to a movie of the kind that causes problems. (HelgeBS didn't have any suitable movies for the moment, unfortunately). If anyone can offer me a small piece of a problematic movie, some 20 seconds or so would be sufficient, I could look into it again.

There were some speculation on another forum that it had to do with a missing or corrupt .meta file, but that seems not to be the case - that should have slightly different symptoms. My guess is that the movie has a format that is not recognized by the current version of ReconstructApSc, and that the .ap file gets too small, maybe empty. My guess, without being able to confirm it, is that removing .ap only, but leaving .sc, will make the movie viewable again.

Anyway, the behavior is not as designed, so if I can have a look at the format, I could at least make ReconstructApSc recognize it and just skip .ap if it can't produce it correctly.

Best Regards
Anders Holst

10

Monday, August 2nd 2010, 8:56pm

I don't have a .meta file either! That is the problem? I have a movie, but it's pretty big (over 5 GBytes). I can create a torrent and send it to you (my upload speed is 8 Mbps so it should transfer in a decent amount of time)

And you are right about .ap file being too small. For example:

I have a .ts movie, size 3.205.252.608 bytes, which have a .ap file of 1.877.568 bytes - this one is OK.

After processing a 5.274.810.000 .ts file, I get a .ap file of 1.242.944 bytes, which seem to be the problem.

This post has been edited 1 times, last edit by "DarthMaul40" (Aug 2nd 2010, 9:05pm)


11

Tuesday, August 3rd 2010, 1:22am

DartMaul40 wrote:

Quoted

I don't have a .meta file either! That is the problem?
No. A corrupt .meta file could be a problem. A missing .meta should not.

Anders

12

Tuesday, August 3rd 2010, 7:09am

DartMaul40 wrote:

Quoted

I don't have a .meta file either! That is the problem?
No. A corrupt .meta file could be a problem. A missing .meta should not.

Anders
If you want, I will cut a few seconds from one of those movies and send it to you. Can you recommend a tool that can cut mpeg2 ts files?

This post has been edited 1 times, last edit by "DarthMaul40" (Aug 3rd 2010, 7:20am)


13

Tuesday, August 3rd 2010, 11:21am

Quoted

If you want, I will cut a few seconds from one of those movies and send it to you.
I tried to send you an email, but maybe it failed.

Yes please, but if you first email me the full .ap file I can check it to find an interesting section to cut out, and email you an appropriate command to cut it out from the movie.

Best Regards
Anders Holst

14

Tuesday, August 3rd 2010, 11:58am

I tried to send you an email, but maybe it failed.

Yes please, but if you first email me the full .ap file I can check it to find an interesting section to cut out, and email you an appropriate command to cut it out from the movie.

Best Regards
Anders Holst
OK, I'll send you the .ap file first.

15

Tuesday, August 3rd 2010, 7:24pm

Delete post

This post has been edited 1 times, last edit by "DarthMaul40" (Aug 3rd 2010, 7:54pm)


16

Saturday, August 28th 2010, 9:48am

I have now analyzed two example movies that I got (thanks DarthMaul40 and HelgeBS). There were in total three different problems with them. Below are the technical details for those who care about such things.

For those who don't, here is the synopsis: Download a new ReconstructApSc with the plugin manager. It should at least make the movies playable. Patches have been sent to DMM for the problems with winding and length calculation.

Details:

One of the movies did not play at all with the .ap file constructed by ReconstructApSc, although it could be played without that file. The movie was SD but very loosely packed so it was large as a typical HD. The problem turned out to be that normally the sequence start markers and picture start markers are in the same ts-frame, whereas in this case they were in consecutive ts-frames. ReconstructApSc has up to now stored the position of ts-frames with picture starts, but apparently it has to store it to sequence starts instead to work properly. I have updated ReconstructApSc to reflect this, and it is available for download the normal way.

But there was another problem too with this movie: After having corrected the .ap-file, it was possible to play it, but not rewind it nor wind faster that x8 forward. For this the .sc-file is used. However, the problem is not with the contents of that file, which is correct, but with the way it is used by enigma2: It jumps to consecutive I-frame picture starts when fast-forwarding or rewinding, but just as above it should jump to sequence starts instead. I have prepared a fix for this and submitted to DMM. It may as usual take some time until it takes effect though.

Now to the second movie. This was a HD movie (or at least it was in the H264 format) but very highly compressed so it was not larger than a typical SD. It played alright, but the problem was that with the .ap-file from ReconstructApSc the displayed movie length was far too short, and the position gauge jumped back and forth during playback. It turned out that this was because the I-frames was very far apart, up to almost 10 seconds simetimes. Currently enigma2 interprets gaps between I-frames over 5 seconds as a "discontinuity" in the movie, and when an .ap-file exists, it is used to find those gaps and compensate for them in the length calculation and position gauge. Compare a situation when this feature works as intended: A movie with a two hour discontinuity in the middle, which without the .ap-file would seem two hours longer than it is, but with the .ap-file this gap is detected and compensated for. However, if up to 10 seconds can occur normally in the movie, the threshold for when it is considered a discontinuity should be increased to (at least) 10 seconds. I have submitted such a patch too to DMM, and again it may take some time before it is applied.

Something I have not done is to make ReconstructApSc construct the .meta file as well when it is missing. This might possibly be done some day in the future.

Best Regards
Anders Holst

17

Saturday, August 28th 2010, 11:54pm

Hey aholst!

since I am using your great tool also in my permanent-timeshift plugin to reconstruct Ap&Sc files when saving a timeshift-record permantly as a movie on HDD, I have one little question/suggestion:

I am using enigma's Task-Manager to execute your binary which also has a progress-bar. But that progress bar only works if the program outputs a % number to see how many percent of a file was done.

Do you think it would be possible to output progress in percent when your binary is creating files? On large *.ts files it takes a very long time to create those files, so it would be nice to know how many % of a file are already done. If it's possible and not too much work, it would be nice if you would consider it for a future version of this tool.

Thanks in advance and great job with that program! :D
Theorie ist, wenn man alles weiß, aber nichts funktioniert.
Praxis ist, wenn alles funktioniert aber niemand weiß warum.