NV_stream_consumer_gltexture_yuv

EGL_NV_stream_consumer_gltexture_yuv

James Jones
Daniel Kartch
Nikhil Mahale
Daniel Koch
Jeff Gilbert

James Jones, NVIDIA (jajones 'at' nvidia 'dot' com)

Complete

Version 4 - November 14, 2017

EGL Extension #94

EGL display extension

Requires EGL_KHR_stream_consumer_gltexture
References EGL_EXT_yuv_surface

This extension makes use of several tokens defined in the
EGL_EXT_yuv_surface extension spec. However support for this
EGLStream extension does not require the EGLSurface extension, or
vice versa. Only the tokens are shared.

The EGL_KHR_stream_consumer_gltexture extension allows EGLStream
frames to be latched to a GL texture for use in rendering. These
frames are assumed to be stored in RGB format and accessed as such
by shader programs. If the producer uses a different color space,
the stream implementation must perform an implicit conversion.

In cases where the producer operates in a native YUV color space, it
may be desirable for shaders to directly access the YUV components,
without conversion. This extension adds a new variant of the
function to bind GL textures as stream consumers which allows
attributes to specify the color space.

None

EGLBoolean eglStreamConsumerGLTextureExternalAttribsNV(
                EGLDisplay       dpy,
                EGLStreamKHR     stream,
                const EGLAttrib *attrib_list)

Accepted as attribute name in <attrib_list> by
eglStreamConsumerGLTextureExternalAttribsNV:

    EGL_YUV_PLANE0_TEXTURE_UNIT_NV                  0x332C
    EGL_YUV_PLANE1_TEXTURE_UNIT_NV                  0x332D
    EGL_YUV_PLANE2_TEXTURE_UNIT_NV                  0x332E

Accepted as attribute name in <attrib_list> by
eglStreamConsumerGLTextureExternalAttribsNV:

    EGL_YUV_NUMBER_OF_PLANES_EXT                    0x3311

Accepted as value for EGL_COLOR_BUFFER_TYPE attribute in
<attrib_list> by eglStreamConsumerGLTextureExternalAttribsNV:

    EGL_YUV_BUFFER_EXT                              0x3300

Call

    EGLBoolean eglStreamConsumerGLTextureExternalAttribsNV(
                EGLDisplay       dpy,
                EGLStreamKHR     stream,
                const EGLAttrib *attrib_list)

to connect one or more texture objects in the OpenGL or OpenGL ES
context current to the calling thread as the consumer(s) of
<stream>. The identity and format of the texture objects used are
determined by <attrib_list> and the current context state.

<attrib_list> must either be NULL or point to an array of name/value
pairs terminated by EGL_NONE. Valid attribute names are
EGL_COLOR_BUFFER_TYPE, EGL_YUV_NUMBER_OF_PLANES_EXT, and
EGL_YUV_PLANE<n>_TEXTURE_UNIT_NV.

If the value of EGL_COLOR_BUFFER_TYPE is EGL_RGB_BUFFER (the
default), then the stream will be connected to a single texture
whose contents are available to shaders as RGB values. If the value
of EGL_COLOR_BUFFER_TYPE is EGL_YUV_BUFFER_EXT the stream will be
connected to some number of planar textures, determined by the value
of EGL_YUV_NUMBER_OF_PLANES_EXT, whose contents are available to
shaders as YUV values. The mapping between YUV values and texture
contents is described in table 3.10.2.1.

If EGL_COLOR_BUFFER_TYPE is EGL_YUV_BUFFER_EXT, the default value of
EGL_YUV_NUMBER_OF_PLANES_EXT is 2. Otherwise it is 0.

                PLANE0            PLANE1            PLANE2
# Planes    Values  Fields    Values  Fields    Values  Fields
--------------------------------------------------------------
    1         YUV     XYZ         unused            unused
    2         Y       X         UV      XY          unused
    3         Y       X         U       X         V       X

            Table 3.10.2.1 YUV Planar Texture Mappings

If EGL_COLOR_BUFFER_TYPE is EGL_RGB_BUFFER, the stream is connected
to the texture object currently bound to the active texture unit's
GL_TEXTURE_EXTERNAL_OES texture target in the current context.

If EGL_COLOR_BUFFER_TYPE is EGL_YUV_BUFFER_EXT, attribute values
must be specified for EGL_YUV_PLANE<n>_TEXTURE_UNIT_NV for all <n>
less than the number of planes. The value of each attribute must
either be a valid texture unit index or EGL_NONE. No two of these
attributes may specify the same valid texture unit index or
reference separate texture units bound to the same texture object.
Plane <n> of the stream contents will be connected to the texture
object currently bound to the indexed texture unit's
GL_TEXTURE_EXTERNAL_OES texture target in the current context, or
will be left unused if the index is EGL_NONE.

Once connected, the stream will remain associated with the initial
texture object(s) even if the texture units are bound to new
textures.

(Note: Before this can succeed a GL_TEXTURE_EXTERNAL_OES texture
must be bound to the appropriate texture units of the GL context
current to the calling thread.  To create a GL_TEXTURE_EXTERNAL_OES
texture and bind it to the current context, call glBindTexture()
with <target> set to GL_TEXTURE_EXTERNAL_OES and <texture> set to
the name of the GL_TEXTURE_EXTERNAL_OES (which may or may not have
previously been created).  This is described in the
GL_NV_EGL_stream_consumer_external extension.)

On failure EGL_FALSE is returned and an error is generated.

    - EGL_BAD_DISPLAY is generated if <dpy> is not a valid,
      EGLDisplay.

    - EGL_BAD_STREAM_KHR is generated if <stream> is not a valid
      EGLStreamKHR created for <dpy>.

    - EGL_BAD_STATE_KHR is generated if <stream> is not in state
      EGL_STREAM_STATE_CREATED_KHR.

    - EGL_BAD_ATTRIBUTE is generated if any attribute name in
      <attrib_list> is not a valid attribute.

    - EGL_BAD_PARAMETER is generated if the value of
      EGL_COLOR_BUFFER_TYPE is not EGL_RGB_BUFFER or
      EGL_YUV_BUFFER_EXT.

    - EGL_BAD_MATCH is generated if EGL_COLOR_BUFFER_TYPE is
      EGL_RGB_BUFFER and EGL_YUV_NUMBER_OF_PLANES_EXT is not 0, or
      if EGL_COLOR_BUFFER_TYPE is EGL_YUV_BUFFER_EXT and
      EGL_YUV_NUMBER_OF_PLANES_EXT is not 1, 2, or 3.

    - EGL_BAD_MATCH is generated if any
      EGL_YUV_PLANE<n>_TEXTURE_UNIT_NV is not specified for any <n>
      less than EGL_YUV_NUMBER_OF_PLANES_EXT, or if it is specified
      for any <n> greater than or equal to
      EGL_YUV_NUMBER_OF_PLANES_EXT.

    - EGL_BAD_ACCESS is generated if any
      EGL_YUV_PLANE<n>_TEXTURE_UNIT_NV is set to anything other than
      a valid texture unit index or EGL_NONE.

    - EGL_BAD_ACCESS is generated if there is no GL context
      current to the calling thread.

    - EGL_BAD_ACCESS is generated unless nonzero texture object
      names are bound the GL_TEXTURE_EXTERNAL_OES texture target
      of each of the appropriate texture units of the GL context
      current to the calling thread.

    - EGL_BAD_ACCESS is generated if more than one planar surface
      would be bound to the same texture object.

    - EGL_BAD_ACCESS is generated if the implementation cannot
      support the requested planar arrangement.

On success the texture(s) are connected to the <stream>, <stream>
is placed in the EGL_STREAM_STATE_CONNECTING_KHR state, and EGL_TRUE
is returned.

When a producer is later connected, if it cannot support the planar
arrangement of the GL texture connection, it will fail with an
EGL_BAD_ACCESS error.

If any texture is later deleted, connected to a different
EGLStream, or connected to an EGLImage, then <stream> will be
placed into the EGL_STREAM_STATE_DISCONNECTED_KHR state.

If the <stream> is later destroyed then the textures will be
"incomplete" until they are connected to a new EGLStream, connected
to a new EGLImage, or deleted.

The function

    EGLBoolean eglStreamConsumerGLTextureExternalKHR(
                EGLDisplay    dpy,
                EGLStreamKHR  stream)

is equivalent to eglStreamConsumerGLTextureExternalAttribsNV with
<attrib_list> list set to NULL.

1.  This competes with GL_EXT_yuv_target as a means for specifying
    how YUV values can be directly accessed by a texture shader
    without conversion to RGB. However, that extension also requires
    a means to render to YUV surfaces in addition to using them as
    textures. Should we go with the approach used here or create a
    GL extension which defines a subset GL_EXT_yuv_target?

    RESOLVED: The extension as is serves immediate needs. Conflicts
    and overlap with other extensions will be addressed if and when
    there is a need to promote to EXT.

2.  This also contradicts how previous extensions for EXTERNAL GL
    textures bind multiplanar surfaces, using separate texture
    objects rather than a single virtual texture object which
    requires multiple texture units. This allows the application
    greater control of the planar arrangement, and the ability to
    leave planes unbound, which may reduce overhead for the
    producer. But it makes applications less portabile if the
    desired arrangement isn't supported.

    RESOLVED: The extension as is serves immediate needs. Conflicts
    and overlap with other extensions will be addressed if and when
    there is a need to promote to EXT.

#4  (November 14, 2017) Mozilla Corporation
    - Const-qualify attrib_list.

#3  (August 19, 2015) NVIDIA Corporation
    - Added enum values.
    - Cleaned up and added contact info for publication.

#2  (May 6, 2015) NVIDIA Corporation
    - Consolidated error codes to make GL interaction simpler.

#1  (April 15, 2015) NVIDIA Corporation
    - Initial draft