IMG_framebuffer_downsample

GL_IMG_framebuffer_downsample

Tobias Hector, Imagination Technologies (tobias.hector 'at' imgtec.com)

Tobias Hector (tobias.hector 'at' imgtec.com)

Complete

Last Modified Date: August 20, 2015
Revision: 11

OpenGL ES Extension #255

OpenGL ES 2.0 or OES_framebuffer_object are required.

This extension is written against the OpenGL ES 3.0.4 Specification
(August 27, 2014).

This extension has interactions with GL_EXT_multisampled_render_to_texture.

This extension has interactions with OpenGL ES 3.1.

This extension has interactions with GL_EXT_color_buffer_float.

This extension has interactions with GL_EXT_color_buffer_half_float.

This extension introduces the ability to attach color buffers to a
framebuffer that are at a lower resolution than the framebuffer itself, with
the GPU automatically downsampling the color attachment to fit.

This can be useful for various post-process rendering techniques where it is
desirable to generate downsampled images in an efficient manner, or for a
lower resolution post-process technique.

This extension exposes at least a 2 x 2 downscale. Other downsampling modes
may be exposed on the system and this can be queried.

No known IP claims.

void FramebufferTexture2DDownsampleIMG(
        enum target, enum attachment,
        enum textarget, uint texture,
        int level, int xscale, int yscale);

void FramebufferTextureLayerDownsampleIMG(
        enum target, enum attachment,
        uint texture, int level,
        int layer, int xscale, int yscale);

Returned by CheckFramebufferStatus:

    FRAMEBUFFER_INCOMPLETE_MULTISAMPLE_AND_DOWNSAMPLE_IMG 0x913C

Accepted by the <pname> parameter of GetBooleanv, GetIntegerv, GetFloatv,
GetInteger64v, and GetInternalFormativ:

    NUM_DOWNSAMPLE_SCALES_IMG                             0x913D

Accepted by the <target> parameter of GetIntegerv, GetInteger64v,
GetIntegeri_v, GetInteger64i_v and GetInternalFormativ:

    DOWNSAMPLE_SCALES_IMG                                 0x913E

Accepted by the <pname> parameter of GetFramebufferAttachmentParameteriv:

    FRAMEBUFFER_ATTACHMENT_TEXTURE_SCALE_IMG              0x913F

Modify figure 4.1, "Per-Fragment Operations.", to add an additional box
"Downscaling" after "Additional Multisample Fragment Operations".

Add a new section 4.1.11, "Downscaling":

    If no multisampling was performed, and downscaling is enabled, fragment
    outputs may be optionally downscaled in a similar way to how multiple
    samples are resolved. If the value of FRAMEBUFFER_ATTACHMENT_TEXTURE_-
    SCALE_IMG is not {1,1}, fragment values are written to an intermediate
    buffer. After all other fragment operations have completed, they are
    then combined to a produce a single color value, and that value is
    written into the corresponding color buffer selected by DrawBuffers. An
    implementation may defer the writing of color buffers until a later
    time, but the state of the framebuffer must behave as if the color
    buffers were updated as each fragment is processed. The method of
    combination is not specified. If the framebuffer contains sRGB values,
    then it is recommended that an average of samples is computed in a
    linearized space, as for blending (see section 4.1.7). Otherwise, a
    simple average computed independently for each color component is
    recommended.

Add the following to Section 4.4.2 "Attaching Images to Framebuffer Objects"
after the paragraph describing FramebufferTexture2D:

    The command

        void FramebufferTexture2DDownsampleIMG(
                enum target, enum attachment,
                enum textarget, uint texture,
                int level, uint xscale, uint yscale);

    allows a rendering into the image of a texture object that has a lower
    resolution than the framebuffer.

    target, textarget, texture, and level correspond to the same
    parameters for FramebufferTexture2D and have the same restrictions.

    attachment corresponds to the same parameter for FramebufferTexture2D,
    but must be COLOR_ATTACHMENTn.

    xscale and yscale are multiplied by texture's width and height,
    respectively, to produce the effective size of the attachment when
    rendering. For example, a texture width of 128 with an xscale of 2 would
    produce a color attachment with the effective width of 256. xscale and
    yscale must be one of the value pairs in DOWNSAMPLE_SCALES_IMG. If the
    xscale and yscale value pair is not available on the implementation,
    then the error INVALID_VALUE is generated.

    The implementation allocates an implicit color buffer for the same
    internalformat as the specified texture, and widths and heights from the
    specified texture level, multiplied by xscale and yscale. This buffer is
    used as the target for rendering instead of the specified texture level.
    The buffer is associated with the attachment and gets deleted after the
    attachment is broken.

    When the texture level is used as a source or destination for any
    operation, the attachment is flushed, or when the attachment is broken,
    an implicit downsample of the color data from the color buffer to the
    texture level may be performed. After such a downsample, the contents
    of the color buffer become undefined.

    The operations which may cause a resolve include:
        * Drawing with the texture bound to an active texture unit
        * ReadPixels or CopyTex[Sub]Image* while the texture is
          attached to the framebuffer
        * CopyTex[Sub]Image*, Tex[Sub]Image*,
          CompressedTex[Sub]Image* with the specified level as
          destination
        * GenerateMipmap
        * Flush or Finish while the texture is attached to the
          framebuffer
        * BindFramebuffer while the texture is attached to the currently
          bound framebuffer.

    Whether each of the above cause a resolve or not is implementation-
    dependent.

Add the following to the sub-section "Attaching Texture Images to a
Framebuffer" after the paragraph describing FramebufferTextureLayer:

    The command

        void FramebufferTextureLayerDownsampleIMG(
                enum target, enum attachment,
                uint texture, int level,
                int layer, uint xscale, uint yscale);

    allows a rendering into a single layer of a texture object that has a
    lower resolution than the framebuffer. It operates like a combination of
    FramebufferTexture2DDownsampleIMG and FramebufferTextureLayer; it allows
    the developer to set scaling values and attaches a single layer of a
    three-dimensional or two-dimensional array texture level.

    target, attachment, level, xscale and yscale correspond to the same
    parameters for FramebufferTexture2DDownsampleIMG and have the same
    restrictions.

    texture can only be a two-dimensional array texture, but otherwise has
    the same restrictions as it does for FramebufferTextureLayer.

    layer corresponds to the same parameter for FramebufferTextureLayer and
    has the same restrictions.

In the sub-section "Effects of Attaching a Texture Image", change the bullet
list to the following:

    * The value of FRAMEBUFFER_ATTACHMENT_OBJECT_TYPE is set to TEXTURE.
    * The value of FRAMEBUFFER_ATTACHMENT_OBJECT_NAME is set to texture.
    * The value of FRAMEBUFFER_ATTACHMENT_TEXTURE_LEVEL is set to level.
    * If FramebufferTexture2D is called and texture is a cube map texture,
      then the value of FRAMEBUFFER_ATTACHMENT_TEXTURE_CUBE_MAP_FACE is set
      to textarget; otherwise it is set to the default (NONE).
    * If FramebufferTextureLayer is called, then the value of FRAMEBUFFER_-
      ATTACHMENT_TEXTURE_LAYER is set to layer; otherwise it is set to zero.
    * If FramebufferTexture2DDownsampleIMG or
      FramebufferTextureLayerDownsampleIMG is called, then the value of
      FRAMEBUFFER_ATTACHMENT_TEXTURE_SCALE_IMG is set to {xscale, yscale};
      otherwise it is set to {1, 1}.

In section 4.4.4 "Framebuffer Completeness", add the following bullet to the
end of the list in subsection "Framebuffer Attachment Completeness":


    * If the value of FRAMEBUFFER_ATTACHMENT_OBJECT_TYPE is TEXTURE and the
      value of FRAMEBUFFER_ATTACHMENT_TEXTURE_SCALE_IMG is supported by the
      internal format of the attachment (see GetInternalFormativ in section
      6.1.15).

In section 4.4.4 "Framebuffer Completeness", add the following bullet to the
end of the list in subsection "Whole Framebuffer Completeness":

    * The value of FRAMEBUFFER_ATTACHMENT_TEXTURE_SCALE_IMG for all
      attachments is {1,1}, or if not, the value of TEXTURE_SAMPLES_EXT or
      RENDERBUFFER_SAMPLES for all attachments is zero.
      FRAMEBUFFER_INCOMPLETE_MULTISAMPLE_AND_DOWNSAMPLE_IMG

Add the following bullet point to the list in Section 6.1.13 "Framebuffer
Object Queries" describing valid pname values when FRAMEBUFFER_ATTACHMENT_-
OBJECT_TYPE is TEXTURE:

    * If pname is FRAMEBUFFER_ATTACHMENT_TEXTURE_SCALE_IMG, then params will
      contain two integer values - the downsample scale pair for that
      attachment.

Change the paragraph in Section 6.1.15 "Internal Format Queries" describing
valid target values to:

    target indicates the usage of the internalformat, and must be one of
RENDERBUFFER, TEXTURE_2D, TEXTURE_CUBE_MAP or TEXTURE_2D_ARRAY.

Add the following paragraphs to Section 6.1.15 "Internal Format Queries" to
the paragraphs describing valid pname values:

    If pname is NUM_DOWNSAMPLE_SCALES_IMG, the number of downscales that
would be returned by querying DOWNSAMPLE_SCALES_IMG is returned in params.
    If pname is DOWNSAMPLE_SCALES_IMG, the available downscale pairs for the
format are written into params.
    Formats that don't support downsampling will still return one valid
downsample scale pair - {1,1}. A value of one for NUM_DOWNSAMPLE_SCALES_IMG
will always mean no downscaling available, as {1,1} must be supported by
every format. Targets that don't support downscaling (e.g. RENDERBUFFER)
will return no downsample scale pairs.

In section 4.4.5 of the OpenGL ES 2.0 Specification "Framebuffer
Completeness", subsection "Framebuffer Attachment Completeness", replace:

    * All attached images have the same width and height.
      FRAMEBUFFER_INCOMPLETE_DIMENSIONS

with:

    * All attached images have the same value of width * xscale and
      height * yscale.
      FRAMEBUFFER_INCOMPLETE_DIMENSIONS

If OpenGL ES 3.1 is supported, replace TEXTURE_SAMPLES_EXT with TEXTURE_-
SAMPLES, and add TEXTURE_2D_MULTISAMPLE to the list of valid targets for
GetInternalFormativ.

If EXT_multisampled_render_to_texture is not supported:
    - ignore references to TEXTURE_SAMPLES_EXT
    - the sample counts returned by GetInternalFormativ with a target of
      TEXTURE* will be the sample values available to be used with
      FramebufferTexture2DMultisampleEXT

If OpenGL ES 3.0 or higher is not supported, ignore references to
glFramebufferTextureLayerDownsample and glGetIntegeri_v.

If either of these extensions are supported, it is not guaranteed that
downscale of these formats is supported, but it may be - users will have to
check with the GetInternalFormat query.

This equally applies to any other additional render formats provided by
extension.

The error INVALID_VALUE is generated if FramebufferTextureLayerDownsampleIMG
or FramebufferTexture2DDownsampleIMG are are called with an <xscale> and
<yscale> value pair that isn't reported by DOWNSAMPLE_SCALES_IMG.

The error INVALID_ENUM is generated if FramebufferTextureLayerDownsampleIMG
or FramebufferTexture2DDownsampleIMG are called with an <attachment> that is
not COLOR_ATTACHMENTn.

Add to Table 6.14 "Framebuffer (state per attachment point)"

                                                    Initial
Get Value         Type        Get Command           Value   Description     Sec.
----------------- ----------- --------------------- ------- --------------- ----
FRAMEBUFFER_-     2 x Z+      GetFramebuffer-       {1,1}   Framebuffer     4.4.2
ATTACHMENT_-                  AttachmentParameteriv         texture scale
TEXTURE_SCALE_IMG

Add to Table 6.35 "Framebuffer Dependent Values"
                                                 Minimum
Get Value         Type        Get Command        Value   Description     Sec.
----------------- ----------- ------------------ ------- --------------- ----
NUM_DOWNSAMPLE_-  2 x Z+      GetIntegerv        2       Number of       4.4.2
SCALES_IMG                                               scale value
                                                         pairs available

DOWNSAMPLE_-      1* x 2 x Z+ GetIntegeri_v      **      Scale value     4.4.2
SCALES_IMG                                               pairs available


** At least {1,1} and {2,2} must be supported as a minimum to support this extension.

GLint xDownscale = 1;
GLint yDownscale = 1;

// Select a downscale amount if possible
if (extension_is_supported("GL_IMG_framebuffer_downsample")
{
    // Query the number of available scales
    GLint numScales;
    glGetIntegerv(GL_NUM_DOWNSAMPLE_SCALES_IMG, &numScales);

    // 2 scale modes are supported as minimum, so only need to check for
    // better than 2x2 if more modes are exposed.
    if (numScales > 2)
    {
        // Try to select most aggressive scaling.
        GLint bestScale = 1;
        GLint tempScale[2];
        GLint i;
        for (i = 0; i < numScales; ++i)
        {
            glGetIntegeri_v(GL_DOWNSAMPLE_SCALES_IMG, i, tempScale);

            // If the scaling is more aggressive, update our x/y scale values.
            if (tempScale[0] * tempScale[1] > bestScale)
            {
                xDownscale = tempScale[0];
                yDownscale = tempScale[1];
            }
        }
    }
    else
    {
        xDownscale = 2;
        yDownscale = 2;
    }
}

// Create depth texture. Depth and stencil buffers must be full size
GLuint depthTexture;
glGenTextures(1, &depthTexture);
glBindTexture(GL_TEXTURE_2D, depthTexture);
glTexStorage2D(GL_TEXTURE_2D, 1, GL_DEPTH_COMPONENT16, width, height);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glBindTexture(GL_TEXTURE_2D, 0);

// Create a full size RGBA texture with single mipmap level
GLuint texture;
glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
glTexStorage2D(GL_TEXTURE_2D, 0, GL_RGBA4, width, height);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glBindTexture(GL_TEXTURE_2D, 0);

// Scale the width and height appropriately.
GLint scaledWidth = width / xDownscale;
GLint scaledHeight = height / yDownscale;

// Create a reduced size RGBA texture with single mipmap level
GLuint scaledTexture;
glGenTextures(1, &scaledTexture);
glBindTexture(GL_TEXTURE_2D, scaledTexture);
glTexStorage2D(GL_TEXTURE_2D, 0, GL_RGBA4, scaledWidth, scaledHeight);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glBindTexture(GL_TEXTURE_2D, 0);

// Create framebuffer object, attach textures
GLuint framebuffer;
glGenFramebuffers(1, &framebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT,
    GL_TEXTURE_2D, depthTexture);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
    GL_TEXTURE_2D, texture, 0);
glFramebufferTexture2DDownsampleIMG(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT1,
    GL_TEXTURE_2D, scaledTexture, 0, xDownscale, yDownscale);

// Handle unsupported cases
if (glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE)
{
    ...
}

// Draw to the texture
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
...

// Discard the depth renderbuffer contents if possible/available
if (extension_supported("GL_EXT_discard_framebuffer"))
{
    GLenum discard_attachments[] = { GL_DEPTH_ATTACHMENT };
    glDiscardFramebufferEXT(GL_FRAMEBUFFER, 1, discard_attachments);
}

/*
    Draw to the default framebuffer using the textures with various post
    processing effects.
*/
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, scaledTexture);
...

1) Should renderbuffers be resolvable in this way too?

   RESOLVED

   No, renderbuffers are considered somewhat legacy and thus will
   not be supported by this extension.

2) Should any scale values other than {1,1} be mandated as minimum?

   RESOLVED

   Yes, {2,2} will also be required. Implementations may support additional
   values though, so a query is also added for other values.

3) What formats support downscaling?

   RESOLVED

   Formats that are guaranteed color-renderable by the core ES 3.1
   specification, excluding integer and signed integer formats, support all
   available downscale modes. Other formats only support {1,1} (no
   downscaling).

4) What should happen if an application calls GetInternalFormativ with a
   target of TEXTURE* (not TEXTURE_2D_MULTISAMPLE)?

   RESOLVED

   For standard OpenGL ES, NUM_SAMPLE_COUNTS should be zero. However, if
   EXT_multisampled_render_to_texture is supported, valid configurations
   for FramebufferTexture2DMultisampleEXT should be returned here.

Revision 1, 2014/08/27
  - First draft

Revision 2, 2015/03/16
  - Mandated {2,2} as a required downsample scale.
  - Coupled x and y scale values into pairs

Revision 3, 2015/03/19
  - Moved framebuffer completeness information to correct (whole framebuffer
    completeness) section, and corrected wording.
  - Added note about minimum support in the overview.

Revision 4, 2015/03/19
  - Added a specific revision of the OpenGL ES 3.0 specification
  - Added an error that only COLOR_ATTACHMENTn can be used as an attachment
    point

Revision 5, 2015/06/02
  - Added internalformat query capability, so that formats can opt into
    downscaling support
  - Added section on downscaling to per-fragment operations.
  - Added issue about what formats support downscaling.
  - Restricted layer downscaling to 2D array textures.

Revision 6, 2015/06/03
  - Fixed typo in incomplete framebuffer condition
  - Added a bullet point to describe the FRAMEBUFFER_ATTACHMENT_TEXTURE_-
    SCALE_IMG parameter to GetFramebufferAttachmentiv
  - Clarified targets to GetInternalFormativ

Revision 7, 2015/06/17
  - Clarified interactions with EXT_color_buffer_float and
    EXT_color_buffer_half_float

Revision 8, 2015/06/22
  - Added NUM_DOWNSAMPLE_SCALES_IMG as a parameter to GetInternalFormativ
  - Added framebuffer attachment incomplete message and removed error when
    scale pair isn't supported by textures' internalformat, as
    internalformat is not necessarily known at attachment time.

Revision 9, 2015/08/19
  - Assigned enum values

Revision 10, 2015/08/20
  - Allowed DOWNSAMPLE_SCALES_IMG to be used with GetIntegerv/GetInteger64v

Revision 11, 2015/12/18
  - Fixed example - "tempScale" is an array so doesn't need to be dereferenced.