Alan Roberts at work
23rd October 2000, 14:53
I've opened the can of worms, and I fully expect you lot to come back at me with a vengence, because, as usual, this is a big topic. I can't cover all of it in one message (time constraints), and some of it would be better with ilustrations, but I can't help that. Anyway here goes.
MY SETUP IS:
Pansonic DX100 camcorder (DV)
Olympus C3030 zoom digital camera
PC, Dell PII/450 with:
extra hard drives for video (2xIBM25Meg on a RAID card)
Epson Stylus Phot 750 6 colour printer
UMAX scanner
Sofware:
Paint Shop Pro
Photo Shop
Premiere 5.1c
EditDV 2
and assorted other bits and pieces.
So, my problem is how to get digital stills from the camera into video, get scans of prints into video, and print digital stills.
VIDEO RESOLUTION AND FORMAT
Video resolution (DV) is 720 pixels by 576 and isn't 4:3. A 4:3 image occupies only 702 of those pixels, the outer 9 from each side are technically called "overscan" and aren't part of the video stream. Also, technically, only 575 lines of video are used, a half line is thrown away from the first and last line of the frame, so that one field starts, and the other ends, in the middle of a line. The reasons for all this are bound up in the history of the development of television, are there's not much future in crying over that. The standards are published by the ITU (International Telecommunications Union), particularly ITU-R BT.470 covers analogue transmission world-wide (the R means Recommendation, BT stands for Broadcast Television) and ITU-R BT.601 covers digitised video, for internal exchange between equipment in studios or between studio centres (it is now enshrined in MPEG2 as well, but more of that anon).
So, when getting stills into video, they have to conform. That means in both resolution and signal format. The resolution of video is rather low by photo standards, and we can get away with it because the opictures move most of the time, and we tend to track moving objects and pay little attention to the rest of the picture. Stills, on the other hand, will be pored over (not poured over as I read in the Guardian last week) and so all parts of the image must bear inspection. So, the rules for taking pictures are not the same as for taking video. But that's another story and belongs in the "Techniques" forum.
Cut to the chase.
HOW I DO IT
So I've got a still picture from my digital camera and I want to use it in video. The process has two major components:
1 reduce the resolution to that of video
2 make the signal confrom to video
Most video editors will do the former, few will do the latter.
REDUCE THE RESOLUTION
Resolution reduction is a filtering operation that any decent graphics package can make a reasonable stab at. Simply import the image into it and resize it to 720*576. But life's not that easy since the pixels of tv aren't square (702/576=1.21875, and not 1.3333...), so I've got to change the pixel shape as well. The best way is to work to a basic shape of 788*576 (no, that's not a typo, I meant 788), inside which is the required 4:3 image of 768*576, and finally squeeze it down to 720*576 for importing into video.
Both Photo Shop and Paint Shop Pro do this quite well, a suitable mixture of cropping the original image and then resizing will get the right aspect ratio. Photo Shop does a marginally better job, but you'll be hard pressed to see the difference, and then only with test pictures.
SIGNAL LEVEL CONFORMANCE
To make the image conform to video rules (and I'm talking of DV in it's native form here) I have to change the dynamic range as well. Graphics images, when stored in 8-bit form, always set black at level zero and white at 255, but tv's different. To allow for inaccurate setup of kit, video black is set at level 16 and white is at 235, only 219 levels used. The reasons for this are extremely good and shouldn't be ignored.
Paint Shop is very good for this, it has a Custom Filter, with a grid of 9 numbers, an offest and a scaler. I set the central number in the grid to 219 (the number of levels I want to end up with), the scaler to 255 (so that it multiply every pixel value by 219/255), and the offset to 16 (to add 16 to every pixel). And the levels are now correct for video. Photo Shop can probaly do it just as well if not better.
PROBLEMS
So, now I've got my still the right shape, and to the right levels. But the result might still not look nice. It certainly won't on a PC, because the contrast range will look wrong, all washed out and lacking in sparkle. However, if I've got a lot of detail in the original image, it can cause what's known as "interlace twitter" when imported into video. This is because tv does not expect every line and every pixel to be capable of being set to any level without knowledge of neighbouring lines/pixels.
BANDWIDTH, RESOLUTION, SAMPLING
The camera has a prefilter between the object (the scene) and the image (on the ccd). On low-cost cameras, it's just the lens, on high end cameras there's an extra optical element put in specifically to perform spatial filtering (birefringent, look it up in an encyclopedia). This limits the amplitude of, or completely removes from the image, those high frequencies that cause problems.
Let's look at horizontals first. With 720 pixel sites, it's tempting to say that we can resolve 720/2=360 cycles of resolution. And that would be true is the samples (pixels) were accuratelt and consistently timed to occur at the peaks and troughs of the sine-wave. But they can equally occur at the zero crossings, where the digital output would be all grey levels. So, we must allow there to be more than 2 samples (pixels) per cycle of resolution. 3 is a good number, more is better. So the resolution limit is more like 720/3=240 cycles per picture width. Trying to get more resolution than this will tend to result in "alias" components. This is what happens when there are less than 2 samples (pixels) per cycle. Under these conditions the spectral contents of the signal are reversed, centred on the sampling frequency, hard to explain in words, easier in pictures. This process, that of misrepresenting one frequency for another, can cause havoc in an image, less so in static ones. It is the cause of wheels going backwards (remember the stage coach wheels in John Wayne pictures?) when temporal sampling is too slow for the motion.
Vertically, the problem's more subtle. Although we have 576 frame lines to put information on, only half of them are there in each field. The frame (25 per second) is a record of two separate images, one 1/50th second after the other. So the only really belong together when the image is stationary and there's little difference between adjacent frames. When there's motion, there's a problem.
When the image is moving, we have to think of each field as standing alone. Thus there's only 288 lines to think of. So the vertical resolution is 576/2=288 cycles per picture height when static, 288/2=144 when moving. In practice, we tend to get away with something in between, and this gives rise to "interlace twitter".
TWITTER
Any vertical frequency present in the scene between 144 and 288 cycles/height will cause twitter. Since the fields come at us at 50Hz, but the frames are at 25Hz, any frequency bewteen 144 and 288 belongs to the frame and so is presented to us at 25Hz, and is visible as a slow twitter. Lower frequencies, below 144 belong to the fields and come to us at 50Hz, seeming steady.
So what?
I have to make sure that my still won't twitter, and that means a filter. Paint Shop and Phot Shop have similar filters (so does Premiere and EditDV etc). They are the one with a 3X3 grid of numbers again. Only this time we don't want to change the signal levels. In Paint Shop, in the extreme, I use:
0 1 0
0 1 0
0 1 0
which adds together, equally weighted, contributions from the centre pixel (the one we're writing to the new, filtered image) and those above and below. I put 3 in the scaler, so that the signal level isn't change, and all the nasty vertical high frequencies go away, and my still won't twitter. Photo Shop is easier, put 0.33333... in the centre column so that they add to 1.
SCANNED IMAGES
All the above applies, but now I've got the vagaries of the print and scanner to contend with. Usually, the black level is high and there's a colour cast after scanning. Again, Paint Shop and Photo Shop can deal with it, so can Photo DeLuxe etc. I normally use Paint Shop, and keep an eye on the RGB histogramme view. It shows the distribution of levels in the image. Then I can tweak Contrast and Brightness to get it to look right on the PC. But, those controls in graphics packages don't work the same way as in video. Video Brightness slides the whole signal up/down by adding to every level, and Contrast changes the range above black level. Graphics Contrast changes the range from centre level (128) and Brightness moves that. So there's a lot of swearing and readjusting before I'm happy. An easier way is to use the 3x3 filter yet again to add/subtract a number to move the black level to about 16, and tweak the multiplier to get the peak white to about 235. Then straight into video.
Enough for now, I've missed out huge chunks, and could have put a lot of this much better, but it's a start.
Now, anybody got any questions?
------------------
alan@mugswellvillage.freeserve.co.uk. Delete village for a spam-free diet.
MY SETUP IS:
Pansonic DX100 camcorder (DV)
Olympus C3030 zoom digital camera
PC, Dell PII/450 with:
extra hard drives for video (2xIBM25Meg on a RAID card)
Epson Stylus Phot 750 6 colour printer
UMAX scanner
Sofware:
Paint Shop Pro
Photo Shop
Premiere 5.1c
EditDV 2
and assorted other bits and pieces.
So, my problem is how to get digital stills from the camera into video, get scans of prints into video, and print digital stills.
VIDEO RESOLUTION AND FORMAT
Video resolution (DV) is 720 pixels by 576 and isn't 4:3. A 4:3 image occupies only 702 of those pixels, the outer 9 from each side are technically called "overscan" and aren't part of the video stream. Also, technically, only 575 lines of video are used, a half line is thrown away from the first and last line of the frame, so that one field starts, and the other ends, in the middle of a line. The reasons for all this are bound up in the history of the development of television, are there's not much future in crying over that. The standards are published by the ITU (International Telecommunications Union), particularly ITU-R BT.470 covers analogue transmission world-wide (the R means Recommendation, BT stands for Broadcast Television) and ITU-R BT.601 covers digitised video, for internal exchange between equipment in studios or between studio centres (it is now enshrined in MPEG2 as well, but more of that anon).
So, when getting stills into video, they have to conform. That means in both resolution and signal format. The resolution of video is rather low by photo standards, and we can get away with it because the opictures move most of the time, and we tend to track moving objects and pay little attention to the rest of the picture. Stills, on the other hand, will be pored over (not poured over as I read in the Guardian last week) and so all parts of the image must bear inspection. So, the rules for taking pictures are not the same as for taking video. But that's another story and belongs in the "Techniques" forum.
Cut to the chase.
HOW I DO IT
So I've got a still picture from my digital camera and I want to use it in video. The process has two major components:
1 reduce the resolution to that of video
2 make the signal confrom to video
Most video editors will do the former, few will do the latter.
REDUCE THE RESOLUTION
Resolution reduction is a filtering operation that any decent graphics package can make a reasonable stab at. Simply import the image into it and resize it to 720*576. But life's not that easy since the pixels of tv aren't square (702/576=1.21875, and not 1.3333...), so I've got to change the pixel shape as well. The best way is to work to a basic shape of 788*576 (no, that's not a typo, I meant 788), inside which is the required 4:3 image of 768*576, and finally squeeze it down to 720*576 for importing into video.
Both Photo Shop and Paint Shop Pro do this quite well, a suitable mixture of cropping the original image and then resizing will get the right aspect ratio. Photo Shop does a marginally better job, but you'll be hard pressed to see the difference, and then only with test pictures.
SIGNAL LEVEL CONFORMANCE
To make the image conform to video rules (and I'm talking of DV in it's native form here) I have to change the dynamic range as well. Graphics images, when stored in 8-bit form, always set black at level zero and white at 255, but tv's different. To allow for inaccurate setup of kit, video black is set at level 16 and white is at 235, only 219 levels used. The reasons for this are extremely good and shouldn't be ignored.
Paint Shop is very good for this, it has a Custom Filter, with a grid of 9 numbers, an offest and a scaler. I set the central number in the grid to 219 (the number of levels I want to end up with), the scaler to 255 (so that it multiply every pixel value by 219/255), and the offset to 16 (to add 16 to every pixel). And the levels are now correct for video. Photo Shop can probaly do it just as well if not better.
PROBLEMS
So, now I've got my still the right shape, and to the right levels. But the result might still not look nice. It certainly won't on a PC, because the contrast range will look wrong, all washed out and lacking in sparkle. However, if I've got a lot of detail in the original image, it can cause what's known as "interlace twitter" when imported into video. This is because tv does not expect every line and every pixel to be capable of being set to any level without knowledge of neighbouring lines/pixels.
BANDWIDTH, RESOLUTION, SAMPLING
The camera has a prefilter between the object (the scene) and the image (on the ccd). On low-cost cameras, it's just the lens, on high end cameras there's an extra optical element put in specifically to perform spatial filtering (birefringent, look it up in an encyclopedia). This limits the amplitude of, or completely removes from the image, those high frequencies that cause problems.
Let's look at horizontals first. With 720 pixel sites, it's tempting to say that we can resolve 720/2=360 cycles of resolution. And that would be true is the samples (pixels) were accuratelt and consistently timed to occur at the peaks and troughs of the sine-wave. But they can equally occur at the zero crossings, where the digital output would be all grey levels. So, we must allow there to be more than 2 samples (pixels) per cycle of resolution. 3 is a good number, more is better. So the resolution limit is more like 720/3=240 cycles per picture width. Trying to get more resolution than this will tend to result in "alias" components. This is what happens when there are less than 2 samples (pixels) per cycle. Under these conditions the spectral contents of the signal are reversed, centred on the sampling frequency, hard to explain in words, easier in pictures. This process, that of misrepresenting one frequency for another, can cause havoc in an image, less so in static ones. It is the cause of wheels going backwards (remember the stage coach wheels in John Wayne pictures?) when temporal sampling is too slow for the motion.
Vertically, the problem's more subtle. Although we have 576 frame lines to put information on, only half of them are there in each field. The frame (25 per second) is a record of two separate images, one 1/50th second after the other. So the only really belong together when the image is stationary and there's little difference between adjacent frames. When there's motion, there's a problem.
When the image is moving, we have to think of each field as standing alone. Thus there's only 288 lines to think of. So the vertical resolution is 576/2=288 cycles per picture height when static, 288/2=144 when moving. In practice, we tend to get away with something in between, and this gives rise to "interlace twitter".
Any vertical frequency present in the scene between 144 and 288 cycles/height will cause twitter. Since the fields come at us at 50Hz, but the frames are at 25Hz, any frequency bewteen 144 and 288 belongs to the frame and so is presented to us at 25Hz, and is visible as a slow twitter. Lower frequencies, below 144 belong to the fields and come to us at 50Hz, seeming steady.
So what?
I have to make sure that my still won't twitter, and that means a filter. Paint Shop and Phot Shop have similar filters (so does Premiere and EditDV etc). They are the one with a 3X3 grid of numbers again. Only this time we don't want to change the signal levels. In Paint Shop, in the extreme, I use:
0 1 0
0 1 0
0 1 0
which adds together, equally weighted, contributions from the centre pixel (the one we're writing to the new, filtered image) and those above and below. I put 3 in the scaler, so that the signal level isn't change, and all the nasty vertical high frequencies go away, and my still won't twitter. Photo Shop is easier, put 0.33333... in the centre column so that they add to 1.
SCANNED IMAGES
All the above applies, but now I've got the vagaries of the print and scanner to contend with. Usually, the black level is high and there's a colour cast after scanning. Again, Paint Shop and Photo Shop can deal with it, so can Photo DeLuxe etc. I normally use Paint Shop, and keep an eye on the RGB histogramme view. It shows the distribution of levels in the image. Then I can tweak Contrast and Brightness to get it to look right on the PC. But, those controls in graphics packages don't work the same way as in video. Video Brightness slides the whole signal up/down by adding to every level, and Contrast changes the range above black level. Graphics Contrast changes the range from centre level (128) and Brightness moves that. So there's a lot of swearing and readjusting before I'm happy. An easier way is to use the 3x3 filter yet again to add/subtract a number to move the black level to about 16, and tweak the multiplier to get the peak white to about 235. Then straight into video.
Enough for now, I've missed out huge chunks, and could have put a lot of this much better, but it's a start.
Now, anybody got any questions?
------------------
alan@mugswellvillage.freeserve.co.uk. Delete village for a spam-free diet.