Square and Non-Square Pixels
By Chris Pirazzi. Thanks to Bob Williams for much of the information.
What Are They?
Pixels in the graphics world are square. A 100 pixel vertical line is the same length as a 100 pixel horizontal line on a
Pixels in an ITU-R BT.601-4 digital video signal (also known as Rec. 601 and formerly CCIR 601) are non-square. A 100 pixel vertical line may be longer or shorter than a 100 pixel horizontal line on a video monitor, depending on the video standard. See below for details.
The term which describes this is pixel aspect ratio. In SGI libraries it is specified as a fraction of vertical (y) pixel size divided by (x) horizontal pixel size. The pixel aspect ratio for square pixels is 1/1.
This term is different than frame aspect ratio, which is a fraction of total vertical (y) frame size over total horizontal (x) frame size, for a given definition of "frame."
Why Do I Care?
Many image processing operations assume a certain pixel aspect ratio. For example, a blur operation with a radially symmetric kernel may
look bad on non-square pixels.
Any rendering of geometry-based graphics to an image must take the pixel aspect ratio into account.
An analog video signal consists of discrete video lines, each of which is a continuous analog signal that represents the luma and chroma for that line of the image. So an analog video signal has no "pixels." When a video board samples a video line, it can sample whatever part of the line it pleases, and it can sample the line at whatever interval it pleases. For a given video signal format (525/625), a board may choose to sample the line according to one of two rules: square sampling or non-square sampling.
A Rec. 601 video signal is non-square. But some video boards have horizontal resampling hardware which will give you square pixels in memory even if you have a Rec. 601 signal.
Therefore, if you get a buffer of pixels from a video board or give one to a video board, you need to know whether your pixels are square or non-square. Among other things, this will determine the image dimensions of your buffer.
How Do I Tell Which Pixel Aspect Ratio I Have?
Any drawing to a graphics monitor using OpenGL and IRIS GL is in square pixels. You can also use SGI graphics boards to process non-square pixels, on-screen or off-screen, by drawing to a framebuffer or pbuffer and then reading back the pixels using
glReadPixels() (or lrectread()).
Images in the DM (dmIC, etc.), MV, or CL libraries may be square pixel or non-square pixel, depending on context. There is a DM parameter DM_IMAGE_PIXEL_ASPECT which specifies the pixel aspect ratio (y/x) in movie files and other situations. Be careful: some of the movie capture and conversion programs shipped with SGI systems even as recently as IRIX 6.3 create movies where this parameter is always 1.0, or is an x/y value instead of a y/x value. These are known problems that will get fixed, but movies may exist with the bogus pixel aspect ratios (see Motion JPEG on SGI Systems for more on that).
For our VL and compression devices,
- vino data from analog input jacks is always square pixel.
- vino data from the IndyCam is always square pixel.
- vino data from the Miranda Mindy 601 Dongle is always non-square pixel.
- ev1 may decode/encode analog video squarely, nonsquarely, or both, depending on the board. Check out ev1 and cosmo1 for the specifics.
- ev1 data from the IndyCam is always square pixel.
- ev1 always inputs and outputs Rec.601 video non-square pixel.
- cosmo1 will use whatever kind of video data the connected ev1 feeds it.
- ev3 is a totally non-square pixel board. it has no analog ins or outs.
- cosmo2 always encodes/decodes analog video squarely. it can also compress or decompress non-square data from memory or from a connected ev3 board.
- sirius lets you choose square or non-square encoding/decoding of analog video signals
- sirius always inputs and outputs Rec.601 video non-square pixel.
- mvp has horizontal resampling hardware, so you can have either square or non-square pixels in memory, regardless of the video signal format. Note: mvp's 625-line square pixels are not completely square. See below.
- divo is a totally non-square pixel board. it has no analog ins or outs.
What Is the Pixel Aspect Ratio of Non-Square Pixels?
For 525-line Rec.601 video, the pixel aspect ratio is exactly 11/10 (y/x).
For 625-line Rec.601 video, the pixel aspect ratio is exactly 54/59 (y/x).
Contrary to popular belief, the pixel aspect ratios are not 4:3, nor are they 720/640 and 768/720! These ratios are defined purely in terms of the pixel sampling frequency of each video standard:
- Rec. 601 digital video is always sampled at 13.5 million pixels per second (for both 525 and 625).
- If you have a 525-line analog NTSC (ANSI/SMPTE 170M-1994) video signal which you want to sample square pixel, the industry standard is to sample at exactly 12 + 27/99 million pixels per second.
- If you have a 625-line analog PAL (Rec. ITU-R BT.470-3) video signal which you want to sample square pixel, the industry standard is to sample at exactly 14.75 million pixels per second.
Therefore, we can derive from this that:
525-line Rec.601 pixel aspect ratio = 13.5 / (12 + 27/99) = exactly 11/10 (y/x)
625-line Rec.601 pixel aspect ratio = 13.5 / (14.75) = exactly 54/59 (y/x)
In theory, we could derive the square pixel sampling frequencies from the frame aspect ratio and other statistics from each analog signal spec. Unfortunately, while both ANSI/SMPTE 170M-1994 and Rec. ITU-R BT.470-3 state that the active picture has a 4:3 frame aspect ratio, neither spec defines "active picture" well enough to use in deriving an accurate square pixel sampling frequency. So the square pixel sampling frequencies in use today are industry conventions, not
stated in any spec.
Wait, How Can That Work?
"OK smarty," you may now be saying, "if the pixel aspect ratio is not 720/640 for 525-line or 768/720 for 625-line, then how can I resolve this with the fact that my images are 720, 640, or 768 pixels wide?" The answer to this is that a 720-pixel-wide non-square pixel image and a 640- or 768-pixel wide square pixel image do not represent the same horizontal range of the underlying signal! Here is an illustration which shows you how to convert between square and non-square pixel
images which have the standard dimensions:
When the diagram says "pad," this means you need to pad the image out to 720 non-square pixels. Usually this means that you start with a greater number of square pixels than 640 or 768. Sometimes those
pixels are not available, and you have to improvise.
When the diagram says either "pad" or "crop," it is best to do the padding or cropping so that the image stays centered.
A video board which is encoding or decoding an analog signal will never need to pad or crop: it can measure or synthesize the continuous analog signal for each video line any way it wants. Each analog video line is long enough to accomodate either horizontal range.
An mvp Quirk
Internally, mvp always encodes or decodes analog and digital video non-squarely. Then, if you have requested square pixel data in memory, mvp digitally resamples each line as it goes into memory or comes out of memory. mvp uses the correct resampling ratios shown above for 525-line video, but for 625-line video, mvp uses 11/12 and
12/11 instead of 54/59 and 59/54, as shown here:
As a result, the 768-pixel-wide images you get from mvp for video-to-memory operations will be horizontally compressed by 1 7/33 pixel, and the 768-pixel-wide images you provide to mvp for memory-to-video operations will be horizontally expanded by 1 5/59 pixel on the video signal (where one pixel is defined as a Rec.601 luminance sampling instant). This quirk only affects square-pixel
625-line data; square 525 and non-square 625 and 525 work correctly.
This problem may be fixed in a future release of the mvp video hardware or system software. Unless you really need to account for this quirk now, we recommend that you assume the mvp hardware implements the correct ratios.