Very Slow Load of Large File

[Chanter]

I am using a USB TV tuner to record broadcast video. It records the broadcast MPEG-2 stream and puts it in a .ts file. I downloaded VideoPad a few weeks ago, and was happy to find that VP can handle this type of file container with no problems, so I bought the software.

 

Yesterday I recorded an HD program (1080i) that runs for 3 hours. The recording file is almost 19 GB long. When I try to load the file, VideoPad begins the loading process. The progress bar shows good progress almost to the half-way point (about 8 seconds). At this point, the progress stops, goes back to the start, and proceeds VERY SLOWLY. I am not sure exactly how long it will take to load, but it appears it will require several hours. I notice that my laptop's processor fan begins running at full speed, putting out a good deal of heat.

 

My laptop is a new HP Envy, Intel i7 processor running at 2.4 GHz, 16 GB of RAM. I have over 1.5 TB of free space on the hard drive. I am using Windows 8.1 (64 bits).

 

Question 1: Do I need more RAM to make things work well?

Question 2: Would splitting the input file into smaller pieces help, even if the final work product output is 2+ hours long?

Question 3: If splitting the input file would help, are there any suggestions for what I might use to split the file?

 

BTW, I am a newbie. For my first "real" project just before the problem above, I successfully loaded and then edited the commercials out of a 1 hour HD program (6.3 GB) with no problems, other than several newbie blunders.

[Chris75]

Chanter what you may want to do first is to clear the unused cache on the program, you may do that by going to Tools and click on Options (preferences on Mac), then click on Disk and click on Clear Unused Cache files. This should help some with the performance issue and this is something you may want to do every so often if you work with large files or if you use the program frequently. You may also click on the check box that says Clear all cache files on Exit so it does not build up.

About your questions:

Question 1: Do I need more RAM to make things work well?

No, what you may need is a dedicated videocard that can handle the video editing. If your computer is using the integrated videocard that means that your processor and RAM will try to share the load.

Question 2: Would splitting the input file into smaller pieces help, even if the final work product output is 2+ hours long?

Yes, it would definitely be better to split and load smaller files since that would take less resources and therefore less time to load. After you have loaded all the files you may create the final file you need. But keep in mind that exporting a large file will take also a long time to finish.

Question 3: If splitting the input file would help, are there any suggestions for what I might use to split the file?

Currently we do not have any program that would allow you to split the file without opening it, but what you may want to do is to check with the program you are using to record if there is an option to split the files automatically so it creates smaller files for you to handle.

[Chanter]

Thanks for the help. I will check with my broadcast TV recorder software. If nothing else, I could program a three-hour recording session as three contiguous one-hour recordings. I need to run an experiment here to see if there is a resulting gap or not.

 

Perhaps my Question 1 could use a bit more discussion. I asked the question about the RAM due the profound difference in loading time between the one-hour (6.3 GB) file and the three-hour program (19 GB) file. The 6.3 GB file loaded in a matter of seconds. I did finally take the time to fully load the 19 GB file. It took about 2.5 hours! Since 19 GB is more than my 16 GB of RAM, I thought that perhaps this huge difference could be explained by the fact that the larger of the two files requires more storage space than I have RAM space.

[c_major]

To extract a video frame for editing, VideoPad has to begin the decoding from the closest keyframe.

 

Some MPEG-2 streams contain very few key frame or some of them have only one keyframe at the beginning. If that's the case VideoPad will try to generate more keyframes in order to have good performance while editing. This process will take a while if it is a long video but it will only need to be done once.

[Chanter]

c_major:

I don't know how to make a determination about keyframes. As I understand it, broadcast TV uses I-frames (keyframes?) quite a bit so that TV receivers can quickly lock (or re-lock) onto signals. The broadcast TV recorder that I am using claims to record the exact MPEG-2 stream as it comes over the air. The signal is strong without problems.

 

However, I did an experiment and discovered something troubling about VideoPad for my application. I loaded the file for the 3-hour TV broadcast (2.5 hours). Next, I immediately put it in the Sequence Bin (no editing whatever). After that, I burned a Blu-ray disc (3+ hours of coding, plus more for time for burning and verifying). Here is what I found:

 

The recorded program on the Blu-ray disc is visibly degraded from the original video file (which I can play on my laptop). The major degradation is that motion is no longer smooth, as if there are skipped frames, or a frame rate conversion. It also appears that the video has lost some detail. For what it is worth, the recorded program occupies about 75% of the space on the disc.

 

My speculation: VideoPad cannot handle a large file directly, so it does lossy compression to make the file smaller (thus the long load time, with the laptop's processor being heavily load and generating a lot of heat all during loading). I could live with this, annoying though it is. But then it appears that editing is done on this compressed file, and this edited, compressed file is then the output put on Blu-ray disc. I would have hoped that the compressed file would be used only as a proxy to define the edits, which then would be applied to the original file.

 

I also did another experiment where I loaded three one-hour files, also from broadcast TV. It took less than 5 minutes to load the 3 hours of video. Once again I did no edits. I simply put them into the sequence bin, one following the other to make a three-hour sequence. I then immediately burned a Blu-ray disc. As with the single 3-hour video, encoding proceeded at appoximately real-time speed. When this disc was ready, I compared it to the original video files. There were subtile differences, mostly loss of smoothness of motion.

 

I am bothered by the large amount time it takes for encoding for Blu-ray disc. I should think it would proceed fairly quickly, since the source file is MPEG-2, and the resulting file for the disc is also MPEG-2. No doubt there is some "repackaging" that needs to be done, but the data should not need to be modified.

 

Given what I see, it seems that VideoPad always re-encodes, even if the input and output are the same format (MPEG-2 in this case). It looks like the re-encoding process is lossy. This is greatest disappoint for me. I was hoping for a lossless process. I guess I need to restart my search for video editing software.

[c_major]

Lossless exporting still under developing. You can turn it on from Options->Export->Enable lossless video exporting. It doesn't support MPEG-2 for the moment but we'll add it in soon.

 

We would like to investigate why the 3 hours recording has problems. Would you mind post here what kind of USB TV tuner are you using?

 

Thanks for report and describe the issue in detail.

[Chanter]

c_major:

I am using a Hauppauge 1191 WinTV-HVR-955Q USB TV tuner. For what it is worth, I have viewed the original video files that I referenced above using both the Hauppauge software and Windows Media Player. Both give me the same result with good video and smooth motion.

 

I am certainly interested in the lossless option you describe. I did see the "button" to select it, but as you point out it does not work yet for MPEG-2. I hope the option appears soon. I would be happy (at least initially) if the option only works for cut-down files (i.e., editing out commercials) even if other more advanced features do not work.

 

BTW, I would be happy to send a Blu-ray data disc with the file in question to anywhere in the world. It would take me way too long to upload 19 GB.

[c_major]

I would be happy (at least initially) if the option only works for cut-down files (i.e., editing out commercials) even if other more advanced features do not work.

 

Yes, lossless export only works for unedited cuts.

 

It would be great if you can send the disc to us. Here's the address for our Canberra office:

 

NCH Software, Inc.

Level 1

19 Barry Drive

Turner, ACT 2601

Australia

 

 

Thank you!

[Chanter]

A Blu-ray disc with the video file is on its way. I printed out this discussion and put it in the envelope along with the disc in order to provide the context when it arrives.

[c_major]

Thank you very much!

 

Should have told you just put "VideoPad Team" on it, sorry.

[c_major]

Hi Chanter,

 

We've received the file. It helped us found a bug which cause the reading process failed in the middle and trigger the converting. We've fixed the bug. Please wait for the next release if it's not urgent. We'll contact you and provide free upgrade to the new version.

 

Also, the video we got from the disc was in 720p 60 fps. I think the TV tuner converted the video (was interlaced 1080p 30 fps). Please try export at 60 fps see if it fix the dropping frame issue.

 

Thanks,

[Chanter]

c_major:

The video format on the disc I sent is true to the original TV broadcast. Your note reminded me that here in the USA, both 720p (60 fps) and 1080i (30 fps) are used as broadcast standards. The TV program I recorded was broadcast by the ABC (American Broadcasting Corporation) Television Network, which uses 720p. The TV tuner recorded exactly what was broadcast, no conversion involved.

 

I will retry exporting at 60 fps instead of 30 fps to see how the video looks.

[c_major]

Please try export to 59.94 fps which is the original frame rate.

[Chanter]

I cannot find any place to control the format of export to Blu-ray disc.

I looked under Tools, Options, Export (tab). All that I find there are check boxes to erase previously recorded discs, verify, automatic disc eject, the experimental lossless disc burning, and automatically fade audio before a transistion. I also checked Tools, Options, Disc (tab). This controls things dealing with the hard disc like file paths and cache control.

I looked under Export, Blu-ray, Blu-ray Movie Disc. This brought up a dialog box where I can enter a disc title and select the path to the disc burner, but nothing else. Hitting the "Create" button to proceed brings up the Create Menu dialog box. Hitting the "Save" button from this dialog box allows me to "Burn It", at which point VideoPad begins encoding with whatever settings it has (which I cannot change, it seems). I tried using the Export Wizard, but all this did is back me up a step or two in the process beginning with "Export, Blu-ray ..." that I oulined above.

I see that I can choose video format if I choose to Export to YouTube (as an example), but I can find no similar selection menu for Export to Blu-ray Movie Disc.

BTW, I am running version 4.14 of VideoPad.

[c_major]

The format for Blu-ray Movie Disc is hidden is intentional. That's because it automatically choose a format that can be played by blu-ray players.

[mx1001]

I have similar problem but never report it. Many times I open my old mini DV captures . They are 1-hour .avi fies 13GB each one (w DV looseless codec 768x576 @25) . With clear cache , progress bar stops even if the vp windows is in focus (no minimized) . Progress bar stops with no certain cause (happens even in i5 with 200GB free space) .

 

And something more . Even when I wait xxxx hours and make some edit job, when I close this project and I open again, the project is not 100% cached . (tried to empty cache again but no success)

[Chanter]

c_major wrote:

The format for Blu-ray Movie Disc is hidden is intentional. That's because it automatically choose a format that can be played by blu-ray players.

and previously:

Hi Chanter,

 

We've received the file. It helped us found a bug which cause the reading process failed in the middle and trigger the converting. We've fixed the bug. Please wait for the next release if it's not urgent. We'll contact you and provide free upgrade to the new version.

 

Also, the video we got from the disc was in 720p 60 fps. I think the TV tuner converted the video (was interlaced 1080p 30 fps). Please try export at 60 fps see if it fix the dropping frame issue.

 

So it looks like the disc I burned is what VideoPad intended to burn, given the current design. The original program material was 720p at 59.97 fps. The resulting disc is the same scan format.

 

I find the results disappointing, especially given that there is still a significant amount of emply space on the disc after burning (I estimate 40%) that could have been used to give higher quality video. And so far as I can tell, there are no controls for me to use to achieve a better result.

 

I will wait to see the result of the bug fix, where there should be no conversion triggered in the middle of the reading process (as mentioned above). I hope the video on the disc will be of higher quality.

[c_major]

I find the results disappointing, especially given that there is still a significant amount of emply space on the disc after burning (I estimate 40%) that could have been used to give higher quality video. And so far as I can tell, there are no controls for me to use to achieve a better result.

 

There is another post here reported low quality blu-ray output as well. We'll look into it.