| Please consider a donation to the Higher Intellect project. See https://preterhuman.net/donate.php or the Donate to Higher Intellect page for more info. |
Definitions: F1/F2, Interleave, Field Dominance, and More
By Chris Pirazzi. Information provided by folks throughout the
company.
Definition of F1 and F2
Interlaced video signals have a natural two-field periodicity. F1 and F2 are the names we give to each field in the sequence. If you look at the waveform of a video field on an oscilloscope, you can tell whether it is an F1 field or an F2 field by the shape of its sync
pulses.
ITU-R BT.601-4 (also known as Rec. 601 and formerly CCIR 601) defines an encoding scheme for digital video.
ANSI/SMPTE 170M-1994 defines Field 1, Field 2, Field 3, and Field 4 for NTSC (figure 7).
ANSI/SMPTE 125M-1992 defines the 525-line version of the bit-parallel digital Rec. 601 signal, using an NTSC waveform for reference. 125M defines Field 1 and Field 2 for the digital signal (figure 4).
ANSI/SMPTE 259M-1993 defines the 525-line version of the bit-serial digital Rec. 601 signal in terms of the bit-parallel signal.
ITU-R BT.470-3 (formerly known as CCIR Report 624-1) defines "first field" (F1) and "second field" (F2) (figure 2) for 625-line PAL.
ITU-R BT.656-2 describes a 625-line version of the bit-serial and bit-parallel Rec. 601 digital video signal. It defines Field 1 and Field 2 for that signal (table I).
We define F1 as an instance of Field 1 or Field 3.
We define F2 as an instance of Field 2 or Field 4.
Field Interlacing and Field Interleave
Generally the term 'interlacing' refers to a technique for signal encoding or display, and 'interleave' refers to a method of laying out
the lines of video data in memory.
The term 'interleave' must be qualified when used in this way. It can also refer to how the samples of an image's different color basis vectors are arranged in memory, or how audio and video are arranged together in memory.
Field Interlacing
The way in which two fields should be interlaced together to produce a picture depends on:
- which field is an F1 field and which field is an F2 field
- whether the fields are from a 525- or 625-line signal.
It does NOT depend on:
- what "order" the F1 and F2 fields are in
- anything relating to field dominance
For 525-line analog signals, the picture should be produced in this manner: (F1 has 243 active lines, F2 has 243 active lines==486 active lines)
field 1 field 2 picture 0-based picture 1-based
******* ******* *************** ***************
(second half only)-----| l.283 0 1
l.21 |----------------------- | 1 2
| -----------------------| 2 3
|----------------------- |-- F2 3 4
F1 --| -----------------------| 4 5
|----------------------- | ... ...
| -----------------------| ... ...
|----------------------- | 483 484
| -----------------------| l.525 484 485
l.263 |------(first half only) 485 486
For official 525-line digital signals, the picture should be produced in this manner: (F1 has 244 active lines, F2 has 243 active lines==487 active lines)
field 1 field 2 picture 0-based picture 1-based
******* ******* *************** ***************
l.20 |----------------------- 0 1
| -----------------------| l.283 1 2
l.21 |----------------------- | 2 3
| -----------------------| 3 4
|----------------------- |-- F2 4 5
F1 --| -----------------------| 5 6
|----------------------- | ... ...
| -----------------------| ... ...
|----------------------- | 484 485
| -----------------------| l.525 485 486
l.263 |----------------------- 486 487
For practical 525-line digital signals, all current SGI VL hardware skips line 20 of the signal, and pretends that the signal has 486 active lines. As a result, we can think of the signal as having exactly the same interlacing characteristics and line numbers as the analog signal: (F1 has 243 active lines, F2 has 243 active lines==486 active lines)
field 1 field 2 picture 0-based picture 1-based
******* ******* *************** ***************
-----------------------| l.283 0 1
l.21 |----------------------- | 1 2
| -----------------------| 2 3
|----------------------- |-- F2 3 4
F1 --| -----------------------| 4 5
|----------------------- | ... ...
| -----------------------| ... ...
|----------------------- | 483 484
| -----------------------| l.525 484 485
l.263 |----------------------- 485 486
For 625-line analog signals, the picture should be produced in this manner: (F1 has 288 active lines, F2 has 288 active lines)
field 1 field 2 picture 0-based picture 1-based
******* ******* *************** ***************
l.23 |--(second half only)--- 0 1
| -----------------------| l.336 1 2
|----------------------- | 2 3
F1 --| -----------------------| 3 4
|----------------------- |-- F2 4 5
| -----------------------| ... ...
|----------------------- | ... ...
| -----------------------| 573 574
l.310 |----------------------- | 574 575
----(first half only)--| l.623 575 576
For 625-line digital signals, the picture should be produced in this manner: (F1 has 288 active lines, F2 has 288 active lines)
field 1 field 2 picture 0-based picture, 1-based
******* ******* *************** ****************
l.23 |----------------------- 0 1
| -----------------------| l.336 1 2
|----------------------- | 2 3
F1 --| -----------------------| 3 4
|----------------------- |-- F2 4 5
| -----------------------| ... ...
|----------------------- | ... ...
| -----------------------| 573 574
l.310 |----------------------- | 574 575
-----------------------| l.623 575 576
All Field 1 and Field 2 line numbers match those in SMPTE 170M and ITU-R BT.470-3. Both of the digital specs use identical line numberings to their analog counterparts. Warning: "Video Demystified" and many chip specs use nonstandard line numbers in some (not all) of their diagrams. Warning: 125M draws fictitious half-lines in figure 3 in very strange places that do not correspond to where the half-lines
fall in the analog signal.
Picture line numbering scheme is shown both 0-based (like the movie library) and 1-based.
Field Interleave
The rules for interleaving two fields in memory depend on the same factors as those for interlacing:
- which field is an F1 field and which field is an F2 field
- whether the fields are from a 525- or 625-line signal.
The rules themselves are identical to those found under "practical
525-line digital signals" and "625-line digital signals" above.
Field Dominance
Field dominance is relevant when transferring data in such a way that frame boundaries must be known and preserved. Examples:
- GPI/VLAN/LTC triggered capture or playback of video data
- edits on a VTR
- interpretation of fields in a VLbuffer for the purposes of interlacing or de-interlacing.
Field dominance can be "F1 Dominant" or "F2 Dominant." It defines
the meaning of a "frame."
For "F1 Dominant," a "frame" is an F1 field followed by an F2 field. This is the protocol recommended by all of the above specifications.
For "F2 Dominant," a "frame" is an F2 field followed by an F1 field. This is the protocol followed by several NY production houses for the 525-formats only.
Most older VTRs cannot make edits on any granularity finer than the frame. The latest generation of VTRs are able to make edits on arbitrary field boundaries, but can (and most often are) configured only to make edits on frame boundaries. Video capture or playback on a computer, when triggered, must begin on a frame boundary. Software must interleave two fields from the same frame to produce a picture. When software de-interleaves a picture, the two resulting fields are in the same frame.
No setting of field dominance should ever affect the temporal ordering of fields in memory. It should only affect the way fields in a temporally increasing order are paired together for one of the above operations.
Thoroughly Ambiguous: Avoid The Terms "Even" and "Odd"
These terms are ambiguous and terribly overloaded. They must be
avoided or carefully defined where used.
"Even and odd" could refer to whether a field's active lines end up as the even scanlines of a picture or the odd scanlines of a picture. In this case, one needs to additionally specify how the scanlines of the picture are numbered (zero-based or one-based), and one may need to also specify 525 vs. 625 depending on the context.
"Even and odd" could refer to the number 1 or 2 in F1 and F2, which is of course a totally different concept that only sometimes maps to the above. This definition seems somewhat more popular.
The VL uses these terms in one place:
- VL_CAPTURE_ODD_FIELDS captures F1 fields
- VL_CAPTURE_EVEN_FIELDS captures F2 fields
The DM_IMAGE_INTERLACING parameter from libdmedia also uses these
terms. That parameter is described below.
Application to MSCs and DMediaInfo->sequence
The VL presents field numbers to a VL application in two contexts:
- For video-to-memory or memory-to-video paths whose VL_CAP_TYPE is
set to VL_CAPTURE_NONINTERLEAVED (fields separate, each in its own
buffer), the functions vlGetFrontierMSC and vlGetUSTMSCPair return
MSCs which count fields (in other VL_CAP_TYPEs, the returned MSCs do
not count fields).
- For any video-to-memory path, the user can use vlGetDMediaInfo to
return the DMediaInfo structure contained in an entry in a VLBuffer.
This structure contains a member called "sequence" which always counts
fields (regardless of VL_CAP_TYPE).
In both of these cases, there should be the following correlation:
- these values should be 0%2 if they represent an F1 field
- these values should be 1%2 if they represent an F2 field
This is a relatively new convention and is not yet implemented on
all devices.
Related Concepts in the SGI Movie and Digital Media Libraries
The movie library (libmoviefile, libmovieplay) and the digital media library (libdmedia) have interleave and dominance concepts as well. You will encounter this when specifying or querying the data format of image tracks in movie files or images you manipulate with the image conversion API (dmIC). The MV/DM interleave concepts are a little different than those for video, because these libraries must handle:
- images with top-to-bottom and bottom-to-top orientation of video lines in memory.
- progressive-scan images (which aren't interleaved at all)
- interleaved video images from a nonstandard crop of the full video raster.
Space, Time, and Memory
This document assumes that display devices scan from the top of the screen to the bottom of the screen. When we say that one line of data is "earlier" or "later" than another, we are referring to the time at which the data is scanned out by a display device. "Earlier" and "later" never refer to the data's address in memory. This document also assumes that pixel data is represented in memory in row-major order (all the x axis pixels for one line are contiguous in
memory).
When presented with an image, these are the key questions:
- What are the spatial/temporal characteristics
of the signal my image represents?
Two choices:- progressive-scan: the spatial ordering of lines from top to bottom is the same as the temporal ordering of lines from earliest to latest.
- interlaced: my image consists of two fields of equal height. The
temporally first field scans through every other line from top to
bottom, then the temporally second field scans through the remaining
lines from top to bottom. The frame parity determines
which field scans which lines:
- even: the spatially highest line of the frame comes from the temporally first field
- odd: the spatially highest line of the frame comes from the temporally second field
This question is completely independent of the image's in-memory
representation. - How is that signal represented in memory?
One choice we can make is orientation:- top-to-bottom: data for spatially lower lines is in higher memory addresses.
- bottom-to-top: data for spatially higher lines is in higher memory addresses.
Another is frame arrangement, useful for images from interlaced signals:
- split field: in memory we have all the lines from one field, and then all the lines from the other field. The data is in some temporal order in memory (forwards or backwards). Orientation is per-field.
- full frame: the fields are weaved together in memory as they would be on a video display to form a complete frame. The data is in some spatial order in memory (upwards or downwards).
In the end, what you really want to know is:
- How do I access memory if I want to proceed through the lines of my image in spatial order, from the top of the image to the bottom of the image?
- How do I access memory if I want to proceed through the lines of the image in temporal order, from the earliest data to the latest data?
For a progressive scan signal, these questions are the same.
Current DM Params
The movie and digital media libraries use a common parameter-value data structure called DMparams to describe image data (see /usr/include/dmedia/dm_image.h). The current parameters will let you
represent most, but not all, of the image information above.
The parameters are DM_IMAGE_ORIENTATION, which can be
- DM_IMAGE_TOP_TO_BOTTOM
- DM_IMAGE_BOTTOM_TO_TOP
and DM_IMAGE_INTERLACING, which can be
- DM_IMAGE_INTERLACED_EVEN
- DM_IMAGE_INTERLACED_ODD
- DM_IMAGE_NONINTERLACED
Examples of Image Parameters in SGI Movie and Digital Media Libraries. These examples may help as you read the
sample code below.
The following sample code shows you how to use DM_IMAGE_INTERLACING and DM_IMAGE_ORIENTATION to derive the four basic properties we defined above: interlaced vs. progressive scan, frame parity, frame arrangement, and orientation. In some cases, these parameters don't provide all the info and we need outside help.
{
unsigned char *image = get_image_data(); /* the image data */
DMparams *imageparams = get_image_params(); /* DM params for image */
int height = dmParamsGetInt(imageparams, DM_IMAGE_HEIGHT);
int bytes_per_line = dmImageFrameSize(imageparams) / height;
int i;
/* our goal is to use DM_IMAGE_INTERLACING and DM_IMAGE_ORIENTATION
* to compute the following four boolean variables:
*/
int signal_is_interlaced; /* signal: interlaced vs. progressive scan */
int even; /* signal: frame parity */
int full_frame; /* in-memory: frame arrangement */
int bottom_to_top; /* in-memory: orientation */
switch (dmParamsGetEnum(imageparams, DM_IMAGE_INTERLACING))
{
case DM_IMAGE_INTERLACED_ODD:
/* implies interlaced signal, odd parity, split-field */
signal_is_interlaced = 1;
even = 0;
full_frame = 0;
break;
case DM_IMAGE_INTERLACED_EVEN:
/* implies interlaced signal, even parity, split-field */
signal_is_interlaced = 1;
even = 1;
full_frame = 0;
break;
case DM_IMAGE_NONINTERLACED:
/* implies full-frame.
*
* DM_IMAGE_INTERLACING==DM_IMAGE_NONINTERLACED could refer
* to either a progressive-scan or an interlaced signal.
* we'll need that information in order to traverse all the
* lines in temporal order.
*/
signal_is_interlaced = detect_whether_signal_is_interlaced();
if (signal_is_interlaced)
{
/*
* DM_IMAGE_INTERLACING==DM_IMAGE_NONINTERLACED also doesn't
* imply a frame parity. we'll need this in order to tell
* which field is temporally first.
*/
even = detect_whether_frame_parity_is_even();
}
full_frame = 1;
break;
default:
error("unsupported DM_IMAGE_INTERLACING");
return;
}
if (!full_frame) /* split field: fields always same size */
assert(height % 2 == 0);
switch (dmParamsGetEnum(imageparams, DM_IMAGE_ORIENTATION))
{
case DM_IMAGE_TOP_TO_BOTTOM:
/*
* with top to bottom orientation,
* - full frame images and each field of split-field images:
* spatially lower (and thus temporally later) lines
* are at higher memory addresses.
* - split-field images:
* the temporally later field is at a higher memory address.
*/
bottom_to_top = 0;
break;
case DM_IMAGE_BOTTOM_TO_TOP:
/*
* with bottom to top orientation,
* - full frame images and each field of split-field images:
* spatially higher (and thus temporally earlier) lines
* are at higher memory addresses.
* - split-field images:
* the temporally earlier field is at a higher memory address.
*/
bottom_to_top = 1;
break;
default:
error("unsupported DM_IMAGE_ORIENTATION");
return;
}
Now, as promised above, we will traverse the lines of the image in spatial order. This can be done entirely with the two DM parameters and no outside help:
for(i=0; i < height; i++) /* traverse spatially from top to bottom */
{
int line;
/* first compute line assuming top-to-bottom orientation */
if (full_frame) /* full frame -- image already in spatial order! */
line = i;
else /* split field representation */
{
if ( even ^ ((i%2)==1) )
line = i/2; /* temporally first field */
else
line = height/2 + i/2; /* temporally second field */
}
if (bottom_to_top) /* adjust if bottom-to-top */
line = height - line - 1;
/* access first byte of that line */
*(image + line*bytes_per_line) = 0xff;
}
Now we'll traverse the lines of the image in temporal order. This requires the outside help:
for(i=0; i < height; i++) /* traverse temporally from earliest to latest */
{
int line;
/* first compute line assuming top-to-bottom orientation */
if (signal_is_interlaced)
{
if (full_frame) /* full frame */
{
if ( even ^ (i >= height/2) )
line = 2*(i % (height/2)); /* temporally first field */
else
line = 2*(i % (height/2)) + 1; /* temporally second field */
}
else /* split fields -- image already in temporal order */
line = i;
}
else /* progressive scan -- image already in temporal order */
line = i;
if (bottom_to_top) /* adjust if bottom-to-top */
line = height - line - 1;
/* access first byte of that line */
*(image + line*bytes_per_line) = 0xff;
}
}
DM_IMAGE_LAYOUT and DM_IMAGE_INTERLACING
An earlier version of the lurker's guide incorrectly documented two parameters, DM_IMAGE_LAYOUT and DM_IMAGE_INTERLACING. DM_IMAGE_LAYOUT described the in-memory frame arrangement (split field vs. full frame), and DM_IMAGE_INTERLACING described whether or not an image was derived from an interlaced signal, and if so, what the frame parity was. This parameterization fully defined all four image characteristics. This is the parameterization used in the dmconvert(1) command-line interface (the "fl" and "il" options), but it was never released as a library API. The name DM_IMAGE_LAYOUT has
actually been used for an entirely different purpose, described in DMbuffers for Video and OpenGL.
Field Dominance in the Movie and Digital Media Libraries
The movie library has no parameter relating to field dominance in any way whatsoever. However, you must make a choice of field dominance when creating a movie file from video data, because a movie file encodes pairs of fields into what it calls "frames," and all edits and
data transfers are on frame boundaries.
Currently, all of the dmIC convertors which can handle video fields are "intra-frame" (processing of one buffer does not require the presence of any other buffers). You send each field to the dmIC convertor in a separate DMbuffer, and dmIC ignores the DM_IMAGE_INTERLACING parameter. Therefore, dmIC currently has no field dominance issue.
Representing Video Images Using the DM Parameters
First we map familiar video concepts onto our four image characteristics:
- Images manipulated in the VL always derive from interlaced signals.
- If your VL images begin on the standard video lines found on the
charts above for analog 525, practical digital 525, analog 625, and
digital 625, then the frame parity is:
- for a signal with F1 dominance (a frame consists of an F1 field
temporally followed by an F2 field),
- for an analog or practical digital 525-line image, odd parity.
- for an analog or digital 625-line image, even parity.
- for a signal with F2 dominance (a frame consists of an F2 field
temporally followed by an F1 field),
- for an analog or practical digital 525-line image, even parity.
- for an analog or digital 625-line image, odd parity.
- for a signal with F1 dominance (a frame consists of an F1 field
temporally followed by an F2 field),
- Images manipulated in the VL always have top-to-bottom orientation.
- The frame arrangement of images in the VL depends on VL_CAP_TYPE:
- VL_CAPTURE_INTERLEAVED: implies "full frame" arrangement
- VL_CAPTURE_NONINTERLEAVED, VL_CAPTURE_EVEN_FIELDS, VL_CAPTURE_ODD_FIELDS: these settings of VL_CAP_TYPE give you one field per entry in a VLBuffer or DMbuffer. This is neither "full frame" nor "split field." When passing such images to MV/DM, the convention is to represent the images as progressive-scan, "full frame" images whose height is equal to the number of lines in the field.
- The VL never gives you the "split field" arrangement (except, as hard-core lurker readers will remember, with ev1, but that was a quirk and will not happen again).
Then, using this mapping, you can determine the correct DM params to use given the rules above.
Representing Motion JPEG Images Using the DM Parameters
Motion JPEG is a way to represent video by JPEG-compressing each field
separately.
When compressing and decompressing each field with dmIC, follow the same convention as for VL_CAPTURE_NONINTERLEAVED above when choosing DM parameters: each field is represented as a progressive-scan, "full frame" image whose height is equal to the number of lines in the field.
When creating a movie file to store motion JPEG, you will be writing pairs of fields at the same time. Therefore, you should create an image track using a DM_IMAGE_INTERLACED_EVEN or DM_IMAGE_INTERLACED_ODD as appropriate.
For more information on the subtleties of Motion JPEG in a Movie Library movie file, see Motion JPEG on SGI Systems.