24p and HD cameras

[hannahsec]

Hi,

I was looking at a Showreel review of the Sony Z1>>>

http://www.showreel.org/memberarea/article.php?12

and I found this:

"As always, you have to remember that video is not capable of one-frame progressive recording in the true film sense, no matter how video camera manufacturers try to suggest otherwise."

Can anyone explain this - as I thought that 24p was the way to record progressive scan 'filmic' images on a video camera?

Do HD cameras such as the Z1 offer this 24p capacity? And if so is it not truly progressive?

Sorry if these questions are answered elsewhere, but I am new to this HD format - though can't afford to buy one at the moment, so will still be using PDX10

Han

[Alan Roberts]

Showreel have it wrong, in a fairly big way. Progressive recording is available ina great many cameras.

In the Panasonic Varicam, HVX200, and the JVD HDV camcorders, the sensors are scanned progressively. The Panasonics then record using DVCProHD and the JVCs record in MPEG2 HDV format. Both manufacturers record this in 720p, Panasonic do it at 50 or up to 60 (depends on the camera and on which continent it was sold) while the JVC HD100 and it's variants record 720p/25 (6 frame GoP MPEG2) and the later 200 and it's variants record 720p/50 (12 frame GoP MPEG2).

All of these are genuine progressive recordings.

Many of the other HD cameras can do progressive at 1080, but only at 24 or 25 or 29.97, or at the NTSC-related speeds. There is no commercially available recorder to record that as it stands, so each frame is recorded to tape as two fields of an interlaced pair, just like film on tv. Since the field-pairs belong to the same wexposure (unlike normal interlace) it's pervfectly possible to extract the original frame from the data stream. Lots of nles do this routinely and have done so for years. It isn't difficult or ground-breaking, just routine.

I'm surprised Showreel should get it that wrong.

[DVdoctor]

This is an excellent example of why we believe the future for product reviews in this market is a collaborative model. If this review were created on DVdoctor in this fashion, Alan would have read it, and have simply corrected the errors, and added his expanded explanation.

IF we can just get some of the DVdoctor regulars to start doing reviews of products this way I believe we can offer to readers a whole new level of value.

Right now if you look at reviews like this one or the one Nigel did, they are all flawed, and the products are beyond the level of the typical single reviewer.

People keep asking for CV to come back and reviews, this is the ONLY way it is going to happen and IMO will offer a far better product. This sites value is in the collective, collaborative expertize of its members.

Can we pick a product and at least try it????

Sharyn

[Alan Roberts]

I'll add some more to what I last said, and in some way it vindicates what Showreel have said, and it may well have been this point that was being made in the first place.

Film for cinema shoots at 24fps. There are several cameras that will shoot at 24fps and many more that claim to but don't.

To shoot at 24fps, you can use a Panavision Genesis, Arri D20, Dalsa, Thomson Viper (I think) but you have to use a separate recorder because these are all plain cameras and not camcorders. Some of the HD camcorders will also shoot at 24, the Sony HDW900 is one, Pansonic Varicam is another. But most of the camcorders that claim to shoot at 24fps, actually don't, they shoot at 23.976 which isn't the same. The speed difference is only 0.1% but that makes a huge difference in production.

For example, the Panasonic HVX200 has 24fps in it's menu system and you can select it. But if you do, the camera system will actually run at 23.976 and you get drop-frame time code. If you do this and then try editing it assuming it's genuine 24, you'll get into some very interesting knots over the time-code. This 23.976 speed is the speed at which NTSC countries actually show 24fps film on tv. Doing it this way permits the "2:3 pulldown" process which generates 2 fields from one film frame, 3 fields from the next (or the slightly better 2:3:3:2 variant) and is fine for them, but for printing to film it can be a real problem because the drop-frame time code arises again. Not only that, but all the sound is shot 0.1% slow and so is not on the expected musical pitch; might appear to be trivial but it isn't because 0.1% isn't musically related so all the sound has to be pitch-changed and that can give unexpected results. BY comparison, the 25fps that we use in the "PAL" countries is 4% fast for film, and 4% is a musical semitone, so all the music comes out a semitone flat if printed onto film, and that aoofends only the musical cognoscenti with perfect pitch who know the music.

So, to recap, "all that's 24 isn't 24". If you want to shoot at 24, you can, but you need to know a lot about the camera and recording system to confirm that it actually is 24 and not somethoing else masquerading as 24.

Hope that helps a bit.

[Alan McKeown]

[QUOTE=Alan Roberts;269940 Not only that, but all the sound is shot 0.1% slow and so is not on the expected musical pitch; might appear to be trivial but it isn't because 0.1% isn't musically related so all the sound has to be pitch-changed and that can give unexpected results. BY comparison, the 25fps that we use in the "PAL" countries is 4% fast for film, and 4% is a musical semitone, so all the music comes out a semitone flat if printed onto film, and that aoofends only the musical cognoscenti with perfect pitch who know the music.

.[/QUOTE]



Alan, I am puzzled as to what you mean by ” 0.1% change in pitch not being musically related while a 4% change in pitch is musically related”.

Surely all the frequencies in the audio signal are changed by 0.1% so all the harmonic relationships remain intact. 0.1% absolute speed accuracy is probably better than many analogue recorders used to achieve and is surely difficult to detect by ear, let alone constitute a problem.

Also, (for an equi-tempered scale) the ratio of two tones a semitone apart is the twelfth root of two, which is 1.0595... which is nearly 6%, a long way from the 4% of the 24 fps to 25 fps speedup.


Alan

[tom hardwick]

Han, it could be that when Showreel wrote: ''video is not capable of one-frame progressive recording in the true film sense'' they meant in the intervalometer sense. I'm scraping the bottom of the barrel slightly, but it's true that even the most basic ciné camera was capable of shooting individual frames spaced apart by accurate intervals, whereas video cameras have struggled to release less than 5 frames at a go, making their timelapse capabilities pretty useless.

tom.

[Alan Roberts]

Tom, time-lapse intervalometer recording is now routine in some HD cameras. Sony HDW750, Panasonic HDX900 and HVX200, Varicam etc, all can do it fine. The tape-based cameras use a video cache of 8-10 seconds, the HVX200 simply records frames as and when necessary.

Alan, the 0.1% pitch change is excruciating to musicians, particularly those with perfect pitch, while the 4% is more acceptable, probably because it's a bigger difference. I used to be a musician and had perfect pitch (50 years ago) but have long lost that facility so I only go on what they tell me these days. Film, shot at 24fps and shown on tv at 25, is generally not pitch shifted, we're used to it. But when shot at 24 and shown in HD at 23.976 or vice versa, pitch-shifting to correct it is routine. The Sony HDCAM decks do this automatically. I don't know what happens in NTSC countries, maybe Sharyn does.

[tom hardwick]

Thanks for the single frame update Alan.

[jbeale]

I notice a number of DVD playback programs can change playback speed (up to 50%) without changing average audio pitch. I believe they chop up the audio and overlap or duplicate bits of it as needed. Obviously the more the speed is changed, the rougher the result. Some audio programs like "Audacity" do a surprisingly good job of this. (I do play music but I'm not among the 0.01% of the US population with Perfect_pitch (http://en.wikipedia.org/wiki/Perfect_pitch)) With a 0.1% speed increase from 23.976 to 24.000 fps I'd think the "chop and recombine while preserving pitch" method would be hard to detect.

I guess this only comes up when shooting on video for output to film. If you shoot 2nd-system sound, production audio recorders like the Aaton Cantar (http://www.aaton.com/products/sound/cantar/index.php) can record with a sampling rate either 0.1% slower or 0.1% faster than 48k (47.952, 48, 48.048 Ks/s) to accomodate later video-to-film or film-to-video transfer with audio at exactly 48k.

[Alan McKeown]

[QUOTE=Alan Roberts;269970]
Alan, the 0.1% pitch change is excruciating to musicians, particularly those with perfect pitch, while the 4% is more acceptable, probably because it's a bigger difference. /QUOTE]


Thanks for the reply, Alan. I’m surprised though; I would have expected it to be the other way round. I thought that the lower limit on absolute pitch accuracy was about 0.3%.


Alan

[Alan Roberts]

Yes, I was surprised too, but the 4% is more readily accepted. Odd, but there you go.

Pitch shifting's a bit more complex than that description though, it involves a resampling process rather than discrete sample manipulation. Best left to the experts.

[hannahsec]

Hi all,

Thanks for all your input, you are really a wealth of knowledge. I still don't think I really understand the difference between HD and HDV or why it is so difficult for a cheaper camera to shoot at 24 fps. I also don't know why HDV is interlaced as I thought that the whole point was to move away from interlacing and work with higher res full frames.

But thanks to you it is all starting to come clearer.

h

[Alan Roberts]

OK, one at a time:

The difference between HD and HDV.

HD is the scanning format, there are two spatial variants:- 1920x1080 and 1280x720. Both come in many field/frame rate variants, but the 720-line version exists only in progressive whereas 1080 can be either progressive at frame rate (e.g. 25) or interlaced at field rate (e.g. 50).

HD conforms to ITU R.BT-709 for its coding equations, while SD conforms to ITU R.BT-601. The reasons for this are purely historic, but do mean that transcoding is always needed along with scaling, when moving between HD and SD.

HD is recorded in many ways, the first was Sony HDCAM on higher-quailty Digibeta tape, it subsamples the video L1440C480 which is known as 3:1:1 and then I-frame compresses to 144Mb/s. The second recording system followed very shortly behind, Panasonic's DVCProHD on standard DVCPro tape but run at higher speed, subsamples video L1440C720 known as 4:2:2 and then I-frame compressed to 100Mb/s. DVCProHD also has a variant for the 720-line system, subsampled L960C480, also 100Mb/s. Then there's the HD D5, HDCAM-SR, and many non-tape systems.

The consumer variant of HD recording is HDV, set up by a consortium of manufacturers, of which Panasonic are not a member. HDV subsamples 108 as L1440x1080C720x540 known as 4:2:2 and then MPEG2 compresses with a 12-frame GoP to 25Mb/s and records on miniDV tape, nice and cheap. HDV also subsamples 720 as L1280x720C640x720 (4:2:2) and MPEG2 compresses 25 or 30 fps to about 19Mb/s with a 6-frame GoP, or 50 or 60 fps to about 25Mb/s with a 12-frame GoP.


Why it's hard to do 24p.

24p isn't hard to do, no harder than any other rate. But it's hard to display. In a NTSC country, where tv runs at 59.94 interlaced, the recorders run at 59.94 interlaced and some cameras will happily run at 23.976fps while recording 2:3 pulldown onto tape. This means you see the dog-log motion that the NTSC countries have to use for film motion, so they're happy with it. But it isn't 24fps, it's 23.976fps. That differece matters because the time-code isn't continuous, it uses the standard NTSC drop-frame time code, where 2 frame codes are missing on every 10th minute. That can be a nightmare in editing.

So, for recording at genuine 24, something else has to happen. The Sony HDW900 does it by running the system at 23.976, 25, 25, 29.97, or 30 progressive, and at 25, 29.97 or 30 interlaced (better known as 50, 59.94 and 60). The problem is monitoring, because 48Hz isn't a standard frequency for many monitors. Other cameras do it as well, but they're all the more expensive ones used for electronic cinema. The Varicam does it neatly by running the tape system continuously at either 59.94 or 60, and then letting you run nthe camera at any speed from 4 to 60. So you can set the tape speed to 60, camera to 24, and it records (and shows in the viewfinder and monitoring) 2:3 pulldown, but you can get a hardware or software playback of only the 24fps frames (or whatever they were recorded at) to get variable speed. Lady Chatterley and Robin Hood were shot this way, plus a whole lot of other stuff.

24 isn't hard to do, it's just hard to cope with the consequences.

Why HD has interlaced options.

In the US, when ATSC was specifying standards for HD back in about 1992, there were 2 competing proposals for the standard: Hollywood wanted 1920x1080 so that they could show off film while the sports/news broadcasters wanted 1280x720 so that they could have fluid motion. There was a limit of about 1.5Gb/s that could be handled for recording, so the 1080 standard had to be limited to 30fps while the 720 standard could go to 60, just right. A 1920x1080 frame has twice the pixels of a 1280x720 frame. Japan was busily watching all this progress, and showed no interest in having progressive tv, they wanted (and still want) only interlace. So the 1920x1080 standard came to have two variants, interlaced and progressive.

At the time, it was also claimed that the consumer decoder4s would be hugely expensive if complex coding were to be used, so the progress variant of 1080 travels via an interlaced structure, just as film does in SD. So, to differentiate between the true progressive of 720 and the true-but-more-complex progressive of 1080, and new term was introduced, psf (progressive with segmented frames). So, much drama shot in 1080 uses the psf variant, where the camera is truly progressive but the frames are handled as field-pairs just like interlace. This gives them the jerky motion that they want because two images are presented to the viewer from each source frame .

There is absolutely no doubt that, if broadcasting were limited to 720p, then those making drama would still want to shoot at the reduced frame rate that gives them the jerky motion, and they'd lose the lovely resolution that they can get.

So, we've got 1920x1080 interlaced, 1920x1080 psf, and 1280x720 progressive.

There's lots more to this, but that should do for now.

[jbeale]

If you were designing entirely from scratch, 24 fps is no harder than 25 or 30 fps, but until recently, video cameras have not used that frame rate. There is a lot of market momentum (from the video chips in the camera, to the installed base of TV receivers and video distribution systems) for traditional NTSC (29.97 fps) and PAL (25 fps) frame rates.

In the USA (only one segment of the world video market, but a significant one) the great majority of TV sets are still standard-def, interlaced-only. I only have a few friends who have HDTVs, and most of those do 1080i (interlaced) if they can do true 1080 resolution at all.

[Alan Roberts]

Just so, 24 is only relevant if you're shooting electronically for printing back to film for the cinema. It has no other purpose. 23.976 is a totally different story though.