mixing interlaced and progressive

[P.Gallagher]

I wanted to ask if it is at all possible to mix the HD footage from my Sony Z1 with the HD progressive footage from the JVC 111 on the same timeline in premiere CS3?

As far as I can figure out both are shot in their own format but are then upscaled to the correct HD ratio on the timeline. I will be using them in PAL format but I think HD in PAL is the same as NTSC, I know so many questions bu I'm really confused.

[dominicwitherow]

I think the JVC only shoots in 720p, where the Z1 only shoots 1080i, so you will have to do some scaling to make the pictures match. Not sure about NTSC land differences, but I have a feeling that there are framerate differences (29 to 25ish), but others here will be better informed on that.

Dominic

[P.Gallagher]

Thanks Dominic

[johnxl1s]

--------------------------------------------------------------------------------

I think the JVC only shoots in 720p, where the Z1 only shoots 1080i,

What would in practical terms be the difference : p and I - 720p and 1080i, i thought p looked better.....ALAN

john

[StevenBagley]

The JVC HD111 only shoots 720p25, meaning that it'll have film-like stocatto motion. The Z1 shoots 1080i25, meaning it has smooth motion. There is certainly no problem in mixing the two on the timeline -- it's done all the time, my last project was a mix of DV, uncompressed SD and 1080i HDVd finishing in SD and editted in FCP. Any editor has been able to do this for years although they require rendering, but certainly FCP6 can do it without.

My suggestion would be to edit in 1080i for two reasons. Firstly, it is becoming the international standard for TV programmes (at least until 1080p50 takes over -- yum). Secondly, downconverting the Z1 footage to 720p25 would introduce serious motion artefacts (as it'd become filmized) and deinterlacing and scaling artefacts. On the other hand, converting the 720p25 footage up to 1080i would only require it to be rescaled.

Steven

[Alan Roberts]

Yes, that advice is all correct. It's FAR better to upconvert the 720 to 1080 than to go the other way round.

I recently did some experiments on this, to find evidence for a Skills Set course. I took a zone plate (phase-space diagram) and upconverted it (2:1) using Corel Paint, Photo Shop and Paint Shop Pro. All conversions were nearly 100% clean. Then I tried downconverting the original (1:2) and all were poor, but Corel was the best by far. None were a patch on doing it the right way (using proper filters in my software, simulating what would be done in a decent hardware downconverter).

When in doubt, upconvert.

[SimonMW]

Could you explain how the downconvert was very poor, and how exactly it would affect viewing from normal viewing distances on a 720p television? Are the artefacts actually visible without zooming in 600% in Photoshop?

Superb down conversion can be done on photos for the web, using various techniques. Not sure why video should be much different.

[Alan Roberts]

The artefacts are plainly visible from 100 yards, let alone zooming inwards.

Down-conversion should always involve the elimination of frequencies from the source image that would cause aliasing in the down-sampled image, because they fall above or near the Nyquist limit. So there should be a low-pass filter before the downsampling, in each direction. Corel Paint does an almost reasonable job of it, but in my experiments both Photo Shop and Paint Shop Pro use simplistic interpolation to generate the down-sampled image, and that does not remove the unwanted frequencies, it merely resamples them, causing highly visible aliasing.

In still images, this is not usually too much of a problem, but in moving pictures it can be a disastter, because the aliased frequencies, when the image moves, move in the opposite direction to the bulk of the image. This is entirely predicted from the normal mathematics of sampling. In video that is then compressed, the disaster comes when the compressor starts looking for motion in order to economise on data rates, because the very edges of the moving objects are confused by these aliased frequencies, causing the compressor to do a poor or bad job.

What you can "superb" down conversion works only because you don't expect the pictures to look good, in practice they're almost always poor to very poor, in my experience. I've yet to see any video on the web that I'd be prepared to pay for, although photos work simply because the aliases don't move (unless you're using stills to create animation, when it can be extremely distracting), but the aliases are there, loud and clear.

[SimonMW]

There must be a solution somewhere. How are all the downconversions from 1080 to 576 for SD transmission being done? Or is it that freeview is so rubbish that the down conversion artefacts get lost in all the other artefacts? ;-)

[StevenBagley]

There must be a solution somewhere. How are all the downconversions from 1080 to 576 for SD transmission being done?

Badly. :) Sony's VTRs can't even get the width of the image right...

Or by using very expensive Snell and Wilcox boxes.

Steven

[infocus]

There must be a solution somewhere. How are all the downconversions from 1080 to 576 for SD transmission being done? Or is it that freeview is so rubbish that the down conversion artefacts get lost in all the other artefacts? ;-)
Stevens sort of answered it already, but the one word answer must be "expensively". ;)

As regards Freeview, the problem is not so much "it's bad", period, as it can be good given a good source. Give it something mediocre, and it'll make it far worse.

[infocus]

The JVC HD111 only shoots 720p25, meaning that it'll have film-like stocatto motion. The Z1 shoots 1080i25, meaning it has smooth motion. There is certainly no problem in mixing the two on the timeline -- it's done all the time,.............
All very true, and this is why it's always best to state (as Steven does) 720p/25 or 720p/50 - rather than just 720p - they are two different matters.

Whilst they can be mixed on the timeline, thee HD111 footage will retain it's 'film-look' motion, whilst the Z1 material will retain the smooth motion appearance. Mixing HD251 material (720p/50) with the Z1s 1080i/25 would be much less noticeable.

[Alan Roberts]

Generally, the broadcasters don't use software down-conversion, Snell&Wilcox make nice boxes that do it better.

Any (almost any) video compression system will work well if you give it sufficient data-rate to work with. The problem arises when the signal gets too complex for the allotted data-rate, that's when you see the warts. The problem with the block-based compressors based on JPEG (all the way up to H.264) is that you see the blocks when it falls over, and the real world isn't made up of blocks, so they stand out. Also, the random detail generated by the marginal compression used for Freeview is a problem, you get all manner of nasty crawly edges.

[P.Gallagher]

Yes John but if you had a second camera man with a Z1 shooting for you in HD can you edit both bits of footage on the same timeline?

Paul

[P.Gallagher]

Sorry Man or Woman shooting second camera for you.....

[Alan Roberts]

Yes, you can edit both 1080 i and 720p/25 or 720p/50 on the same timeline. Certainly Edius can do that, and I believe Prep pro can as well. What we're saying is that you'll get the best image quality if the project editing is done at 1080 rather than 720.

[SimonMW]

With regard to aliasing, this will be a major factor for Red, since projects shot with S35mm DOF characteristic will need to be shot 4k and then downconverted to 1080 for HD output. So there would need to be a careful balance between that and keeping the image sharp after downconversion.

[Alan Roberts]

It's a major factor in the design of the RED, yes. But, I'm sure they know that, and will have taken the correct action. BBC R&D has one on order, and I'll run it through my mangle when they get it, and aliasing will be one of the things I'll be looking for. I'm pretty certain I won't find any though, because the design team knows what it's doing.

As far as I can tell, the camera works in 4k all the time, and internally down-converts to whatever standard it outputs, and I'd be amazed if tghat downconversion isn't done well, remember they've not got interlace to worry about.

[infocus]

As far as I can tell, the camera works in 4k all the time, and internally down-converts to whatever standard it outputs, ...........
I'm not sure about that (at least the "all the time" bit) - I thought it had modes for sub 4k standards where it just used part of the chip - the logic being that then such as standard 2/3" lenses could be used. If it always did 4k, even when a lesser format was being output, it would always need full coverage lenses. Obviously, that may be the desired option sometimes for DOF reasons.

[StevenBagley]

I'm not sure about that (at least the "all the time" bit) - I thought it had modes for sub 4k standards where it just used part of the chip - the logic being that then such as standard 2/3" lenses could be used. If it always did 4k, even when a lesser format was being output, it would always need full coverage lenses. Obviously, that may be the desired option sometimes for DOF reasons.

There is a windowed 2k mode (but you've still got the same problem of needing to subsample the data to HD resolutions) -- I'm sure Graeme will be along shortly to fill in the details (but the impression I got talking to him at IBC was that they are very interested in making sure everything is done properly).

Steven

[nattt]

Yes, there are 2k and 4k modes, where the 4k takes in the whole sensor and is currently limited to a max of 30fps recording, and the 2k windowed centre of the sensor mode where you can currently record up to 72fps.

Whenever we're downconverting, we're taking special care over aliasing issues.

Graeme

[Alan Roberts]

What I meant was that I think the sensor is always scanned at full resolution (and that's not the same as full pixel-count, the pixels are 5microns apart just as in 2"/3 HD). If the image is taken from a smaller window of the sensor, then it would still be sampled at the same resolution as at 4k, just a smaller portion of it. So no down-conversion needed, just a redistribution of the existing samples in real time. No problem. However, if the image is wanted at, say, 2k with the same metric dimensions as it makes at 4k, then the camera would work at 4k and do a downconversion on the sensor data. There are several ways to do this, the simplest would be to build a different decoder for the Bayer pattern for each output pixel-count, generating the output format directly from the sensor, the other is to generate the 4k signal and downonvert it properly. I'm sure that Graeme knows all this and has done what makes best sense in the camera.

[nattt]

Yes, that is correct Alan.

Graeme

[Alan Roberts]

There you go then :D

[infocus]

..........the 2k windowed centre of the sensor mode where you can currently record up to 72fps.
And in this mode, what is the active area of the sensor - or more exactly what lens is required in 2/3", 1/2" etc terms?

[P.Gallagher]

You all have me totally confused now with all the tech stuff.

The answer to my question is YES you can edit both different types of footage if need be but using 2 cameras using 1080 interlaced footage leaves it all the more simple.

Thank you all for your help, yet again this forum has proved more than helpful.

Paul

[Alan Roberts]

Well, the pixels are the same size as in 2"/3 video, 5 microns, and the windowed 2k resolutuon, being half 4k, will make for an image 12mm wide, same as super16.

[Rob James]

Yes, that advice is all correct. It's FAR better to upconvert the 720 to 1080 than to go the other way round.

I recently did some experiments on this, to find evidence for a Skills Set course. I took a zone plate (phase-space diagram) and upconverted it (2:1) using Corel Paint, Photo Shop and Paint Shop Pro. All conversions were nearly 100% clean. Then I tried downconverting the original (1:2) and all were poor, but Corel was the best by far. None were a patch on doing it the right way (using proper filters in my software, simulating what would be done in a decent hardware downconverter).

When in doubt, upconvert.

Alan, a couple of thoughts occur to me. You mentioned "Proper filters in my software". Perhaps there is a product opportunity for you here both for a Photoshop plug-in or stand alone image scaler but maybe also for a software standards converter since nobody else is doing one with decent filters.

[Alan Roberts]

Agreed, there's an opportunity. But, I've no idea how to go further with it. In my software (written in BBC BASIC for Windows, 47k lines of code, 1.6Megs) I calculate filters ad hoc. Filtering takes ages, even on a fast dual core laptop. If anybody's interested in taking this up as a product, I'd be very happy to collaborate, providing the mathematical input to it. Maybe Graeme Nattress would find it interesting, he's good at coding and has some splendid plugins for Premier.

[Rob James]

I sincerely hope somebody takes it up. Graemme, are you listening? With dual quad cores I cannot believe that the crunching time is insurmountable. As you've said before, it's the filters that are crucial.

I'm guessing this is slightly analagous to the audio A to D converter problem. I.e. you need an absolute brick wall filter and the snag with that is that, if such a thing existed, the delay would be infinite. Implementing it in software is also processor time-consuming hence oversampling etc.

[Alan Roberts]

Spot on, you need filters to suppress the frequency content that will cause trouble in the output before you generate the output. With a brick wall filter, not only is the delay near infinite, but you get enormous ringing in the pass-band unless you design the filter right, which makes it even longer. The trick is to work out how best to compromise, because sheer science isn't always the best way to go, Richard Russell (now retired from BBC R&D, writer of the code for BBC BASIC for Windows, and highly experienced designer of very good image processing kit) uses shorter filters than I do, because he's used to designing hardware implementations, and so he has to be aware of the cost implications.

[Alan Roberts]

Graeme's busy right now ;) but he's thinking about it.

[Rob James]

Hopefully you can nab him when he gets home. ;)

[Alan Roberts]

I'm way ahead of you :D

[nattt]

Yes, busy indeed. I am thinking of it though. Just getting ready for a certain trade show that starts Monday....