Please consider a donation to the Higher Intellect project. See https://preterhuman.net/donate.php or the Donate to Higher Intellect page for more info. |
Difference between revisions of "Displaying Video Using Galileo (ev1)"
(Created page with "<p>By Michael Portuesi.</p> <h3>Introduction</h3> <p>Because the VL supports so many different pieces of video hardware, you might be tempted to think that for a given task,...") |
|||
Line 11: | Line 11: | ||
<p>With most video devices that support a full rate video path to | <p>With most video devices that support a full rate video path to | ||
memory, you can create a video to memory path and display the video in | memory, you can create a video to memory path and display the video in | ||
− | your application using OpenGL. Your application wakes up whenever the | + | your application using [[OpenGL]]. Your application wakes up whenever the |
transfer buffer is full, retrieves a frame from the buffer, and draws | transfer buffer is full, retrieves a frame from the buffer, and draws | ||
the frame to the display.</p> | the frame to the display.</p> | ||
Line 154: | Line 154: | ||
displayed in an XmMainWindow widget. If the XmMainWindow needs to make | displayed in an XmMainWindow widget. If the XmMainWindow needs to make | ||
the menu bar span two rows instead of one, it will move the widget | the menu bar span two rows instead of one, it will move the widget | ||
− | displaying video lower on the screen to | + | displaying video lower on the screen to accommodate the fatter menu |
bar. As a Motif programmer, you have no control over this behavior; | bar. As a Motif programmer, you have no control over this behavior; | ||
your only option is to recognize it and react accordingly.<br> | your only option is to recognize it and react accordingly.<br> | ||
Line 543: | Line 543: | ||
[[Category:SGI]] | [[Category:SGI]] | ||
+ | [[Category:Programming]] | ||
+ | [[Category:Graphics]] |
Latest revision as of 01:39, 1 September 2020
By Michael Portuesi.
Introduction
Because the VL supports so many different pieces of video hardware, you might be tempted to think that for a given task, such as displaying incoming video on the workstation screen, that you may write one piece of code for all the devices VL supports. Unfortnately you cannot, and nowhere is this more apparent than when you are programming video display using Galileo (ev1) and its relatives.
With most video devices that support a full rate video path to memory, you can create a video to memory path and display the video in your application using OpenGL. Your application wakes up whenever the transfer buffer is full, retrieves a frame from the buffer, and draws the frame to the display.
Galileo does not offer you this option; its video to memory path
does not support a full-rate stream into memory. Instead, it offers a
VL screen node. You create a video to screen path using the
VL, set some controls on the path to tell the screen node the X window
it should use and where the video should be located, then call
vlBeginTransfer()
to display the video to the user.
The code to display video using ev1 differs from other devices not only in the path setup, but in the maintenance work your application must perform. Using ev1, the display work is handled automatically; your application need never service a transfer buffer. But your application must keep the video positioned and sized appropriately, and update its position if the user moves the window. This task is the central topic of this document.
The ev1 Screen Drain Nodes
The ev1 video device (in all its forms) provides the programmer with three screen drain nodes. Here's how they are described by vlinfo:
lipstick.esd.sgi.com % vlinfo name of server: number of devices on server: 2 device: ev1 1 nodes = 14
[entries omitted]
Screen Drain A, type = Drain, kind = Screen, number = 0 Screen Drain B, type = Drain, kind = Screen, number = 1 Screen Drain C, type = Drain, kind = Screen, number = 2
[entries omitted]
However, in practical use, you will likely use only the first two nodes - numbers 0 and 1. Node 1 can be divided into two half-sized 12-bit screen displays. When you divide screen drain node 1, the second of the two half-displays is denoted by node number 2. This special "half size" mode is not discussed in this document, and judging from the restrictions on its use over and above those discussed here, it seems inappropriate at best for general-purpose application software.
Positioning Video, or Why Does videoin Jump All Around the Screen?
By far, the biggest headache facing the ev1 programmer is simply positioning the video at the right place on the screen. The video drain doesn't render into the framebuffer, as does all other visual display on the system. Instead, the ev1 device provides a hardware-generated "overlay" which is merged with the contents of the frame buffer. As such, all attributes of the video display including screen position and size are controlled through the VL, rather than through X or OpenGL.
To make matters worse, the ev1 hardware imposes a raft of restrictions on where and and under what circumstances the video overlay can be positioned. None of them are particularly friendly to the application developer. Here is a list of the problems:
- The coordinates for the video overlay are expressed using screen
(root window) coordinates, not the coordinate system of the window
containing the overlay.
- You cannot position a video overlay such that it shares the same
raster lines with another video overlay that may already be on the
screen. Because the other video overlay might belong to another
application, you must deal with this problem in your code if your
application might possibly be run in tandem with any other video
applications.
If your application attempts to move the video overlay to a position where it conflicts with another, the overlay will jump to an arbitrary screen position which does not cause a conflict. Additionally, the VL will not return an error to your application. It is your application's responsibility to check for and deal with this problem explicitly
This problem, and one solution to it, can easily be seen by running the bundled videoin utility. If you start up two instances of videoin for ev1, and try to position them on the screen, the windows will often jump to random locations when you drag their window frames. videoin contains code to check theVL_ORIGIN
control immediately after it sets it, and then moves its X window to where the video landed! See below for advice on how to handle this situation in a better, if not totally, acceptable way.
- You cannot position the video overlay even partially offscreen. If
you attempt to do so, the edge of the video overlay will "bump up
against" the edge of the screen and go no further.
- You cannot position the video overlay such that any portion of the
overlay occupies the last seven pixels on right hand side of
display. If you attempt to do so, the right edge of the video overlay
will "bump up against" pixel 1271 on the display and go no
further.
- To determine if video was positioned properly in all these
instances, you must immediately get the
VL_ORIGIN
control after setting it and check to see if it is in the same spot you asked for.
How to Position Video Properly in a Motif application
A civilized application program must allow its windows to go where
the user places them. We can forgive videoin due to its
simplicity. But a better solution for most applications is to put the
window where the user wants it, but to post a notifier if the ev1
video display cannot be placed to match the window location. This
might disappoint the user, but nevertheless it still leaves them
feeling like they have control over the windows on their screen.
- The first step is to determine when the user moves the topmost
window of your application to another place on the screen. This is
done by installing an Xt event handler for the appropriate events
on the topmost window of your application. When the user moves the
window, the X server sends an event to the window, and Xt will then
invoke a callback function in your code.
There is one major complication. Not only must you install an event handler on the topmost shell widget, you need to install one on every widget in the tree between the shell and the widget displaying video. This reason for this complication is because Xt and Motif geometry operations can move the video window as well.
The most subtle form of this problem happens when the video is displayed in an XmMainWindow widget. If the XmMainWindow needs to make the menu bar span two rows instead of one, it will move the widget displaying video lower on the screen to accommodate the fatter menu bar. As a Motif programmer, you have no control over this behavior; your only option is to recognize it and react accordingly.
Here is a code sample to install event handlers all the way up the widget hierarchy, starting with the widget displaying video and ending with the application's top-level shell widget:void installStructureNotifyHandler() { if (handlerActive) return; Widget shell = _WidgetDisplayingVideo; while (!XtIsShell(shell)) { XtAddEventHandler(shell, StructureNotifyMask, FALSE, &structureNotifyHandler;, (XtPointer) this); shell = XtParent(shell); } XtAddEventHandler(shell, StructureNotifyMask, FALSE, &structureNotifyHandler;, (XtPointer) this); handlerActive = True; }
The code to undo this mess looks very similar:void removeStructureNotifyHandler() { if (!handlerActive) return; Widget shell = _WidgetDisplayingVideo; while (!XtIsShell(shell)) { XtRemoveEventHandler(shell, StructureNotifyMask, FALSE, &structureNotifyHandler;, (XtPointer) this); shell = XtParent(shell); } XtRemoveEventHandler(shell, StructureNotifyMask, FALSE, &structureNotifyHandler;, (XtPointer) this); handlerActive = False; }
The event handler callback must check for the appropriate event type and react accordingly:
void structureNotifyHandler(Widget w, XtPointer client_data, XEvent* event, Boolean*) { switch (event->type) { case ConfigureNotify: // we've been moved around onscreen. update our video display if (XtIsRealized(w)) { // // figure out dimensions and screen coordinates of the // video widget // Position x, y; Position rootx, rooty; Dimension widgetW, widgetH; XtVaGetValues(w, XmNx, &x;, XmNy, &y;, XmNwidth, &widgetW;, XmNheight, &widgetH;, NULL); XtTranslateCoords(XtParent(w), x, y, &rootx;, &rooty;); // // armed with root x and root y, we now calculate the // placement of the video overlay. // if (!displayPath) return; // determine size of video display. VLControlValue val; vlGetControl( svr, displayPath, src, VL_SIZE, &val; ); int sizeW = val.xyVal.x; int sizeH = val.xyVal.y; // divide by current ZOOM setting. assume that zoom // control is 1/n. vlGetControl( svr, displayPath, drn, VL_ZOOM, &val; ); sizeW /= val.fractVal.denominator; sizeH /= val.fractVal.denominator; // // find the origin coordinates of the video overlay, // taking into account that the video overlay cannot // go even partially off the screen. // int originX = rootx + ((Position) (widgetW / 2) - (Position) (sizeW / 2)); int originY = rooty + ((Position) (widgetH / 2) - (Position) (sizeH / 2)); // did we go off the screen? if so, make adjustments. Dimension screenWidth = WidthOfScreen(XtScreen(w)); Dimension screenHeight = HeightOfScreen(XtScreen(w)); if (originX < 0) { // left edge originX = 0; } if (originY < 0) { // top edge originY = 0; } if (originX + sizeW > screenWidth) { // right edge originX = screenWidth - sizeW - 1; } if (originY + sizeH > screenHeight) { // bottom edge originY = screenHeight - sizeH - 1; } // place the video overlay. val.xyVal.x = originX; val.xyVal.y = originY; vlSetControl(svr, displayPath, drn, VL_ORIGIN, &val;); // verify the video was placed where we want it. checkVideoOverlayLocation(originX, originY); } break; default: break; } }
- Finally, your code for checking
VL_ORIGIN
shouldn't just check to see that the origin is different than you asked - it should try to determine whether the problem is due to overlapping video, or bumping against the edge of the display. You may not want to complain to the user or take drastic action if the video is merely running against the edges of the screen.
void checkVideoOverlayLocation(Position originX, Position originY) { VLControlValue actualVal; vlGetControl( svr, displayPath, drn, VL_ORIGIN, &actualVal;); // // Collisions between video overlays occur only when they are // on the same raster lines (horizontal), so comparing just the // y-value is sufficient to detect these cases. // // We do not worry about offscreen placement, // because that is taken into account elsewhere. // if (actualVal.xyVal.y != originY) { // // video overlay didn't position properly. Your app should // take some sort of action here. Keep in mind that if you // post an error dialog, you should also set a state flag so // that you do not post the dialog more than once. This // code will (unfortunately) get called multiple times, // because the event handler for ConfigureNotify events is // installed on several widgets in the hierarchy. // } // // Compare the x values to determine // if the right edge of the display // cannot go past pixel 1271 // if (actualVal.xyVal.x != originX) { // your app might want to do something here } }
Sizing Video
- When you set the
VL_ZOOM
control, you also need to set theVL_SIZE
control at the same time to guarantee the video displays properly.
Get Out of Jail For Free (almost)
Yet another unpleasant side effect of the ev1 video overlay are the "jailbars" which appear when the window is larger than the video, or when the user moves around on the screen a window containing ev1 video.
The "jailbars" happen when you assign an X window to the
video node by setting the VL_WINDOW
control on the
path. The graphics hardware ignores the framebuffer contents for that
region of the display, replicating the last value that was in its
hardware registers. That last value is usually the pattern for the
window manager border, which when replicated looks like a row of brass
bars on the display.
If the window is larger than the video overlay, the regions of the window above and below the video overlay which don't contain video information will display jailbars. There is no way around this. There are a few ways to deal with the problem:
- Make sure the widget or window containing the video is never
larger than the video display itself. Sometimes, this is
easier said than done.
- A second way to deal with the problem is most applicable in
situations where you may not have absolute control over the window
size (perhaps your code might be part of a library which is used by
several applications). You put the widget containing video together
with two other child widgets inside an XmForm. The two child widgets
"cover up" the jailbar area above and below. The parent Form
widget moves and resize them horizontally as the window moves and
resizes. You must move and resize them vertically at the same time
you place the video overlay.
void createJailBarWidgets() { // // top widget // topArea = XtVaCreateManagedWidget ( "topArea", xmDrawingAreaWidgetClass, form, XmNtopAttachment, XmATTACH_FORM, XmNbottomAttachment, XmATTACH_NONE, XmNleftAttachment, XmATTACH_FORM, XmNrightAttachment, XmATTACH_FORM, XmNrightPosition, 0, XmNtopOffset, 0, XmNbottomOffset, 0, XmNleftOffset, 0, XmNrightOffset, 0, XmNwidth, topAreaWidth, XmNheight, topAreaHeight, XmNbackground, BlackPixelOfScreen(XtScreen(form)), (XtPointer) NULL ); // // bottom widget // bottomArea = XtVaCreateManagedWidget ( "bottomArea", xmDrawingAreaWidgetClass, form, XmNtopAttachment, XmATTACH_NONE, XmNbottomAttachment, XmATTACH_FORM, XmNleftAttachment, XmATTACH_FORM, XmNrightAttachment, XmATTACH_FORM, XmNtopPosition, 0, XmNbottomPosition, 0, XmNrightPosition, 0, XmNtopOffset, 0, XmNbottomOffset, 0, XmNleftOffset, 0, XmNrightOffset, 0, XmNwidth, bottomAreaWidth, XmNheight, bottomAreaHeight, XmNbackground, BlackPixelOfScreen(XtScreen(form)), (XtPointer) NULL ); } void placeJailBarWidgets(Dimension widgetW, Dimension widgetH, Position originX, Position originY, Position rootY, Dimension videoH) { Dimension bottomAreaWidth, bottomAreaHeight; Dimension topAreaWidth, topAreaHeight; bottomAreaWidth = topAreaWidth = widgetW; int newTopHeight = ((int) originY) - ((int) rooty); if (newTopHeight < 0) { topAreaHeight = 0; } else { topAreaHeight = newTopHeight; } int newBottomHeight = ((int) widgetH) - (((int) originY) - ((int) rooty) + ((int) videoH)); if (newBottomHeight < 0) { bottomAreaHeight = 0; } else { bottomAreaHeight = newBottomHeight; } XtVaSetValues(topArea, XmNheight, topAreaHeight, (XtPointer) NULL); XtVaSetValues(bottomArea, XmNheight, bottomAreaHeight, (XtPointer) NULL); }
- Another way to handle this inside a Motif program is to nest the widget displaying video inside two XmForms. The widget is attached positionally to the edges of the innermost XmForm, and the innermost XmForm is attached relatively to all edges of the outermost XmForm. We do not present code for this solution here, but it is a simple matter for any moderately experienced Motif developer.
IMPACT Graphics do not display jailbars when used with ev1. But unless you know your application will run only on IMPACT, you need to worry about jailbars.
Finally, once a window is used with the ev1 video display node, it cannot be used for anything else. You cannot render into the window using X or GL drawing operations, even if you destroy the ev1 path which was displaying into the window. Once a window is in jail, it's a life sentence. The only surefire way to "get out of jail" is to destroy the window and create one anew.
Why Can't I See the Video?
There is a very common problem encountered by people who try to
display ev1 video within an X or Motif program. If you call
vlBeginTransfer()
before the window intended to
display it is fully created and mapped (shown) to the screen, the
video will not display.
This problem occurs because the X window system is
asynchronous. The X server does not immediately create and display
windows when the application invokes the appropriate calls. Instead,
the requests are queued up for the server, which performs them at
some arbitrary point in the future. If you start the transfer
immediately after calling a function like XtManageChild()
or XMapWindow()
, the video display won't work.
There are two ways to deal with the problem.
- If you are writing an Xt or Motif application, or have access to
the X event loop, set an event handler on the widget to display video,
using the
StructureNotifyMask
. When you receive aMappingNotify
event, the X server has mapped the window to the display, and it is safe to start the video transfer.
You can code this easily by adding a new clause to theswitch
statement in theStructureNotifyHandler
sample function shown above. This is the preferred, though the more complicated, method of dealing with the problem.
- If you are writing a non-Motif X application where you do not have
access to the X event loop, try calling
XSync()
after you make the call to display the window. This will force all outstanding requests to be sent to the X server, andXSync()
will wait until the server has processed them.
Handling Signal Preemption
When your code receives a VLStreamAvailable
event for
the display path, your code should do the following:
- call
vlSetupPaths()
to set the path up once again:
vlSetupPaths(svr, &displayPath, 1, VL_SHARE, VL_SHARE);
- set the
VL_ORIGIN
,VL_SIZE
andVL_ZOOM
controls for the path, since the previous app using this path have left it in an unknown state.
- If you have windows mapped to cover the jailbars (see "Get
Out of Jail For Free," above), update their size and
position.
- call
vlBeginTransfer()
to restart the video transfer:
vlBeginTransfer(svr, displayPath, 0, NULL);