Please consider a donation to the Higher Intellect project. See or the Donate to Higher Intellect page for more info.

Tips for Writing Your Main (select, event) Loop

From Higher Intellect Vintage Wiki
Jump to navigation Jump to search

By Chris Pirazzi and Michael Portuesi.

The VL source code examples we ship demonstrate several possible ways to write one's main event dispatching loop, none of which are sufficient for typical real applications. This is because typical real applications:

  • need to react to events from more than just video. They need to wake up to service AL audio queues, serial ports for deck control, and sometimes input or GUI events from the X server.
  • do not wish to spin unnecessarily. When no video or other events are occuring, the applications do not wish to use up CPU cycles.
  • cannot use the rather bizzare and video-centric vlMainLoop() mechanism, either because they need control of the main select() themselves, or they do not have control of the main select() (as is the case with the equally bizzare and X-Centric XtMainLoop()).

In additon, there are some outright bugs in some of our sample programs that happen to surface only very rarely in those programs, but may surface more commonly in real app code.

This document shows you how to roll your own main event loop in a way which will navigate you around the most common pitfalls of VL programming, and in a way which will not yank control of the main select() loop away from you. Then it shows you how to deal if you are unfortunate enough to be stuck with XtMainLoop().

This document will assume that your app creates a VL path involving memory. Much of what is said here is useful even if that is not the case.

Some of the tips described here depend on whether you are using the classic VL buffering API or the O2 VL buffering API. Check out What Are the SGI Video-Related Libraries for information on what that means.

Use Your Own select() Loop

You must avoid these pitfalls in writing a select() loop:

  • Use vlGetFD() and vlBufferGetFd()

    The VL offers two different file descriptors:

    1. The "event" file descriptor, returned by vlGetFD() (which is the same as vlConnectionNumber()), unblocks when there is a VL event available.
    2. The buffer file descriptor, returned by vlBufferGetFd(), unblocks when there are one or more items in an input buffer, or one or more slots in which to enqueue items in an output buffer. See below for a more precise definition.

    You should generally select() on both file descriptors. VL events are useful for notifying your app of path-global events such as transfer failures, control changes, GPI triggers, and VLPath sharing events. However, when you need something to wake you up as soon as possible when data or space becomes available, the buffer fd is far superior. The buffer fd tends to fire much sooner than the corresponding VLTransferComplete event would arrive. This is partially because the UNIX kernel event that makes the VL buffer fd fire goes right from the video driver to the app, whereas the VLTransferComplete event from the driver has to take a trip through video daemon (on some systems) before it gets to the app.

    On some systems such as O2, the event fd and the buffer fd may be the same. If you write code that assumes the file descriptors are different, it should work on all implementations (FD_SET and poll() will do the right thing).

    If you are using the O2 buffering API in a memory-to-video program, you may also be interested in the fd returned by dmBufferGetPoolFD(). See below for more information.

  • Classic Buffering API: Don't Bother with VLTransferComplete Events

    Because the buffer fd is a superior source of data/space available events, there really isn't any reason to include VLTransferComplete events in your VLEventMask (unless of course you have no VLBuffer in your path). Some code we have seen uses VLTransferComplete and VLSequenceLost events to try and detect exactly which items were dropped or duplicated when the VLBuffer overflows or underflows. This method will not work on all devices---VLTransferComplete and VLSequenceLost are only indications that one or more items transferred or failed to transfer. See the section on "detecting dropped or duplicated data" below for a cleaner and more portable way to detect dropped or duplicated data.

    If you are using the O2 buffering API for a video-to-memory path, you receive data as part of VLTransferComplete events. Because there is no daemon, event reception is as efficient as using the buffer file descriptor on other systems. So it's a good idea to field VLTransferComplete events for transfers to and from memory.

  • The VL Event FD is Twisted---You Must Use vlPending() First

    On some systems, the file descriptor returned by vlGetFD() has a bizarre and undocumented behavior when used in a select() loop. This is the same behavior exhibited by the IRIS GL file descriptor returned by and documented in qgetfd() (qtest() is vlPending() in this context): ensure the proper behavior of select, call qtest before
    and after you call select. When select indicates data is
    available on the file descriptor, there may not be GL event
    data to read.  Similarly, there is a condition where select
    would block, but data is available. The use of qtest before and
    after select guards against this...
    Below, we will present a code example which heeds this warning.

  • Process the Right Amount of Data At Each Wakeup

    Classic buffering API:

    We have seen VL code which assumes that it will receive one VLTransferComplete event for each incoming item of data. We have seen other code that assumes that exactly one item of data/space is available every time the buffer fd unblocks. Neither of these strategies will yeild correct code on all VL devices. Both VLTransferComplete and the buffer fd are indications that one or more items of data/space are available.

    An example: say you're doing video-to-memory, and you process 1 item per VLTransferComplete event. You may get a VLTransferComplete event which "counts" more than one input item, but you have only processed one input item. Each time this happens, the fill level of your input buffer will grow, until eventually your input buffer is permanently overflowing.

    O2 buffering API:

    For memory-to-video, there is always one item of data for each incoming VLTransferComplete event. The video-to-memory situation is as with the classic buffering API: each VLTransferComplete could indicate one or more free spaces available in the pool. Either way, since other events (VLTransferComplete or not) can fill the event queue, you still cannot assume that exactly one item of data or space is available each time you wake up on the buffer/event fd.

    Both buffering APIs:

    Therefore, every time you wake up, your N-item buffer may contain anywhere between 0 and N valid (video-to-memory) or free (memory-to-video) items. It is a good idea to process all of those items before going to sleep again. Here are the ways you can do that:

    • The best and most race-free method is to ask how many valid/free items are available once when you wake up, and then process that many items.
      • A video-to-memory application can determine the number of valid items available with vlGetFilled() (classic buffering API) or vlGetFilledByNode() (O2 buffering API). This provides a lower bound on the number of times you can successfully perform a buffer operation (vlGetNextNextValid() or vlEventRecv() of VLTransferComplete).
      • A memory-to-video application using the O2 buffering API can determine the number of free items available in the video pool using dmBufferGetPoolState(3dm). This provides a lower bound on the number of times you can successfully call dmBufferAllocate. A memory-to-video application using the classic buffering API cannot directly determine the same statistic. N-vlGetFilled() will not do the trick. So move on to the next method.

    • The next best method is to simply keep performing the buffer operation (vlGetNextValid(), vlGetNextFree(), vlEventRecv(), dmBufferAllocate()) and processing the resulting items until the buffer operation no longer succeeds. This method is a little racy: while the application is processing valid/free items, more items may arrive, and so the application may end up processing data and not calling select() for long periods of time. For many applications this is a serious problem, because other parts of the application (such as the GUI) need to run as part of the select() loop every once and a while. In fact, if items arrive every M milliseconds, and the application takes longer than M milliseconds to process each item on average, then the application will infinite loop in its video event handling code!

      The way around this is simply to impose some finite limit on the number of video items the application processes per loop. A good limit to choose is N.

    The code below shows the best methods in action.

Here is some sample code using the classic buffering API which illustrates these points in action:

  int vlFD, vlBufferFD;

  /* open VL */
  /* set up VL path */
  /* create and associate VLBuffer of size "buffer_size" items */
  /* vlSelectEvents: mask out VLTransferComplete/VLSequenceLost events */
  /* vlBeginTransfer */

  vlFD = vlGetFD(vid->vlSvr);
  vlBufferFD = vlBufferGetFd(vid->rb);
  while (1)
      int i;

      /* first check to see if VLEvents are available */
      while (vlPending(vid->vlSvr))
          VLEvent ev;
          VC(vlNextEvent(vid->vlSvr, &ev;));
          switch (ev.reason)
            case VLTransferFailed:    
              ... /* transfer has now stopped.  deal with that. */

            /* ... other events ...*/

      /* now check to see if we have some data/space available */
      if (path_is_video_to_memory_path)
          int nfilled = vlGetFilled(vid->vlSvr, vid->buf);
          for(i=0; i < nfilled; i++)
              VLInfoPtr info;
              info = vlGetNextValid(vid0>vlSvr, vid->rb);
              assert(info); /* filled count sez there must be data */

              ...; /* use the data! */

              vlPutFree(vid->vlSvr, vid->rb);
      else if (path_is_memory_to_video_path)
          /* classic VL buffering API has no "vlGetFillable()" operation */
          /* process at most N items */
          int nfillable = buffer_size;
          for(i=0; i < nfillable; i++)
              VLInfoPtr info;
              info = vlGetNextFree(vid0>vlSvr, vid->rb);
              /* we're not sure exactly how many times we'll succeed */
              if (!info)

              ...; /* use the space! */

              vlPutValid(vid->vlSvr, vid->rb);

      /* now do the select() */
        fd_set readset, writeset, exceptset;
        /* video */
        FD_SET(vid->vlFD, &readset;);
        FD_SET(vid->vlBufferFD, &readset;);
        /* do FD_SETs for other agents: audio, serial, X, gfx, ... */
        if (-1 == select(getdtablehi(), &readset;, &writeset;, &exceptset;, NULL))

      /* now service the events that woke up the select() */
        /* at this point, we don't even have to call FD_ISSET() on any 
         * of the video fd's if we don't want to---vlPending and 
         * vlGetFilled will tell us all FD_ISSET() would and more.

        /* act on FD_ISSET for other agents: audio, serial, X, gfx, ... */

One thing to notice is that upon entering this main loop for the first time, we call vlPending() before we select() for the first time. This is crucial. As explained by the obscure paragraph from qgetfd(3G) quoted above, there are circumstances where select() could hang forever even though VL events are available. We chose to also put the call to vlGetFilled() before select() just because it was convenient. There could very well be data/space in the buffer before the first select(), but in this case the buffer fd would unblock immediately, so it's not a big deal.

How to Get the Same Effect with XtMainLoop() in the Way

As any Motif programmer knows, Xt owns the main loop for your application, and it calls select() for you. You have little control over that. However, it is still possible for an Xt program to respond to VL events. It is also possible to handle VL events which may already be in the queue before XtMainLoop() calls select(). Finally, you can even do all of this in a ViewKit program, without munging VkApp:run() (ViewKit's wrapper around the Xt event loop).

To respond to VL events, you must register Xt callbacks for the VL event queue, and also for the transfer buffer, if your application uses one. For each, you provide a callback handler function. Here is some sample code to set up the callbacks.

void setupVLEvents()
    // register an Xt callback for the VL's event queue.
    // EventHandlerFunc is an Xt input callback which will
    // process VL events.
    videoInputId = XtAppAddInput( appContext,
                                 (XtPointer) XtInputReadMask,
                                 (XtPointer) clientData);

    // Register an Xt callback for the file descriptor associated
    // with the VL buffer.  We do this instead of asking for
    // TransferComplete events.
    // BufferFullHandlerFunc is an Xt input callback which will
    // take the next frame from the buffer and display it on the
    // screen.
    videoBufferId = XtAppAddInput(appContext,
                                 (XtPointer) XtInputReadMask,
                                 (XtPointer) clientData);

    // Choose the VL events we want to listen to.  Your application
    // might have a different set of events it cares about than the
    // ones shown in this example.
    vlSelectEvents(svr, displayPath,
                  VLDeviceEventMask |
                  VLDefaultSourceMask |
                  VLStreamPreemptedMask |
                  VLStreamAvailableMask |
                  VLControlChangedMask |
                  VLControlPreemptedMask |
                  VLControlAvailableMask |

void removeVLEvents()
    if (videoInputId != NULL) {
        videoInputId = NULL;
    if (videoBufferId != NULL) {
        videoBufferId = NULL;

A sample callback handler for VL events might look something like this:

void EventHandlerFunc(XtPointer client_data, int*, XtInputId*)

void EventHandler()
    VLEvent ev;
    while (vlPending(svr)) {
        vlNextEvent(svr, &ev;);

        switch (ev.reason) {
        case VLTransferComplete:
        case VLTransferFailed:
        case VLDeviceEvent:
        case VLDefaultSource:
        case VLStreamPreempted:
        case VLStreamAvailable:
        case VLControlChanged:
        case VLControlPreempted:
        case VLControlAvailable:
            printf("unhandled VL event, reason %d\n", ev.reason);

The callback handler for the VL transfer buffer looks something like this:

void BufferFullHandlerFunc(XtPointer client_data, int*, XtInputId*)

void BufferFullHandler()
    VLInfoPtr info = NULL;
    info = vlGetLatestValid(svr, buf);

    if (info != NULL) {
        // do something with the video data.  Here, we draw it to the
        // screen.
        void* dataPtr = vlGetActiveRegion(svr, buf, info);
        glRasterPos2i(originX, originY);
        glPixelZoom(1.0, -1.0);
        glDrawPixels(sizeW, sizeH,
                     GL_ABGR_EXT, GL_UNSIGNED_BYTE, dataPtr);
        vlPutFree(svr, buf);

To make sure you properly handle VL events which may already queued before select is called, you should explicitly call the Xt event handlers immediately after starting the transfer on the path:

void startVideo()

    // ... other VL setup code ...


    // start video transfer.

    vlBeginTransfer(svr, displayPath, 0, NULL);

    // do a first pass check to handle any
    // queued events from the VL or the transfer buffer


When Exactly Does the Buffer FD Unblock?

Classic buffering API:

  • For a VLBuffer in a video to memory path, the fd unblocks if there are one or more items which have been placed there by the device (using vlPutValid()) but have not yet been snagged by the application (using vlGetNextValid()). This is equivalent to the condition where vlGetFilled() >= 1.

  • For a VLBuffer in a memory to video path, the fd unblocks if there are one or more spaces which have been made available by the device (using vlPutFree()) which have not yet been snagged by the application (using vlGetNextFree()). There is no vlGetFillable(), but if there were, this would be equivalent to the condition where vlGetFillable() >= 1. It is not equivalent to the condition where (buffersize-vlGetFilled()) >= 1. Don't be surprised if you see an inconsistency between that quantity and the behavior of the buffer fd for a memory to video buffer.

O2 buffering API:

  • The buffer fd and the event fd are identical: they both unblock if there are one or more VLEvents which have been deposited by the device but not yet snagged by the application (using vlEventRecv()).

  • For memory to video applications, the fd returned by dmBufferGetPoolFD() is also useful. It unblocks if one or more spaces are available for allocation in the DMpool (using dmBufferAllocate()).

Caveat about Spinning, Especially for High-Priority Processes

One tricky thing about writing any program that uses select(), and in particular one that uses the VL buffer fd, is avoiding spinloops. Spinloops are accidental circumstances where your program is essentially always executing, usually because select() is always returning immediately. Spinloops are bad because they burn away CPU power on the basic overhead of processing thousands of select() system calls per second. Spinloops are really bad for non-degrading high-priority processes (see npri(1), schedctl(2), or Seizing Higher Scheduling Priority): since such processes want to run all the time, and since they run at a higher priority than all normal processes (including the X server), they essentially lock up the system, rendering it completely useless while

the program is spinning.

The caveat: say your app wants to wait until there are N (N>1) items in an input buffer, and then dequeue them all at once. For example, it may want to coalesce buffers for disk writing or interleaving. Say your app selects on the buffer or event fd. After the first item arrives in the VL buffer, these fds will always unblock immediately, and your app will begin to spinloop. At the moment, the only recourse offered to a VL app is not to select() on the buffer fd when there are already items available (which you can tell from vlGetFilled()), and instead to provide select() with a timeout on the order of one field time. This is rather gross. Another gross solution is to call vlGetNextValid() or vlGetNextFree() as soon as any data or spaces become available, keeping track of the resulting VLInfoPtrs yourself. To the VL, each time you call select(), there is no data or no space, so your app will block for a reasonable amount of time waiting for the next data/space. When you have accumulated N VLInfoPtrs, you act on the data.

A similar thing happens if your app wishes to wait until there are N spaces in an output buffer. Since there is no vlGetFillable(), you have to use the second technique above---grab free data immediately and keep track of VLInfoPtrs yourself.

The real solution to this problem has been in the AL and other libraries for some time---fillpoints (see ALsetfillpoint(3)). Perhaps someday we will have fillpoints in the VL.

This caveat also applies to the O2 buffering API. The buffer/event fd and the dmBufferGetPoolFD() fd will unblock immediately if their condition is true. The dmBufferSetPoolSelectSize() function lets you set the threshold of free DMpool space for unblocking, but this function was not implemented as of this writing (the select size was fixed at one item).

Detecting Dropped or Duplicated Data

When an input buffer overflows (the application fails to dequeue data fast enough), or an output buffer underflows (the application fails to enqueue data fast enough), or there is some device failure,

video frames can be dropped on input and duplicated on output.

The VL events VLTransferComplete and VLSequenceLost indicate that one or more items of video data transferred or failed to transfer. There is not (for all VL devices) a one-to-one correspondence between these events the individual fields/frames that transferred or failed to transfer. Therefore, these VL events are not useful for determining exactly which items were dropped or duplicated.

For video to memory paths with the classic buffering API, the easiest way to detect dropped data is to look at the "sequence" field of the DMediaInfo struct returned by vlGetDMediaInfo() for a particular item of video data (a particular VLInfoPtr). Every VL device keeps a field counter which increments by one every time a field crosses an input jack of the machine (whether or not that field is successfully transferred into memory). When a transfer is successful, the VL will snap the current value of the sequence value and place it in the DMediaInfo struct of that particular VL item (for frames you get the DMediaInfo struct for the first field of the frame).

For video to memory paths with the O2 buffering API, you can retrieve similar timestamp information from a DMbuffer using dmBufferGetUSTMSCpair(). In this case the counter value has the units of MSC, which may be fields or frames depending on VL_CAP_TYPE (see vlGetFrontierMSC(3dm)).

Therefore, by watching for jumps in the "sequence" or MSC values of successive pieces of video data, you can detect all failed transfers, you can tell exactly where they fell in the sequence of data, and you can tell exactly how many fields were dropped. This is the recommended method for detecting drops on input.

At the moment, there is no equivalent queue of DMediaInfo or USTMSCpair structures coming back to the application in a memory to video path. Perhaps one day there will be.

But it turns out that there is a second mechanism (part of the UST/MSC implementation in the VL), called vlGetFrontierMSC(), which can be used to determine all the same statistics for both input and output in a symmetric manner. The mechanism works identically for both buffering APIs. See vlGetFrontierMSC(3dm) to find out how to detect all the failed transfers and the length of all the failures. You can also use vlGetFrontierMSC(3dm) to determine exactly which video items were dropped or duplicated, but this is more tricky and not currently explained in vlGetFrontierMSC(3dm).