Browse Source

libobs-opengl: Treat pixel format 0 in IOSurface as 32BGRA

Per the Apple documentation, when 0 is returned from
IOSurfaceGetPixelFormat, the IOSurface is invalid. Prior to [1] however,
any invalid surfaces were (likely accidentally) treated as 32BGRA.
And as it turns out, any IOSurface received from Syphon does not have
the pixel format set, thus returning 0. Because the pixel format happens
to always be 32BGRA, this was never caught. [1] then slightly refactored
the code, fixing the bug not realizing it ever existed. This lead to
Syphon no longer working.

Treating 0 as 32BGRA appears to be the only way to fix this situation
for now and restore prior behavior. Ideally in the future Syphon
provides valid IOSurfaces. An upstream fix to Syphon-Framework is
already submitted, however even if merged, servers would need to update
to that fixed version before things work again. Therefore, this
workaround is needed on the OBS side.

However, the plan for now is for the Metal renderer to not get this
relaxation. It never had it (so Syphon never worked there, making this
not a regression) and it's marked as experimental. Hopefully, by the
time it's stable and/or default, Syphon will be fixed and servers have
been updated.

[1]: 53ad05db9fa07ede1578787b9c3c30031f597a92
Sebastian Beckmann 1 month ago
parent
commit
2d5ba21204
1 changed files with 7 additions and 3 deletions
  1. 7 3
      libobs-opengl/gl-cocoa.m

+ 7 - 3
libobs-opengl/gl-cocoa.m

@@ -482,15 +482,19 @@ gs_texture_t *device_texture_create_from_iosurface(gs_device_t *device, void *io
 
             break;
         }
+            /* Versions of Syphon before 2dc6d31 provide IOSurfaces with pixel format 0.
+             * A pixel format of 0 means they are invalid, but in practice the IOSurfaces provided by Syphon are always
+             * 32BGRA. Historically, OBS had a bug that accidentally treated any invalid surface as 32BGRA, so those
+             * invalid surfaces were never noticed. To keep Syphon working, we need to keep this (wrong) behavior.
+             * This is suboptimal as IOSurfaces that are actually invalid (possibly coming from somewhere else) also
+             * get treated as valid 32BGRA, but it is how it is. */
+        case 0:
         case kCVPixelFormatType_32BGRA: {
             color_format = convert_gs_format(GS_BGRA);
             internal_format = convert_gs_internal_format(GS_BGRA);
             texture_type = GL_UNSIGNED_INT_8_8_8_8_REV;
             break;
         }
-        case 0:
-            blog(LOG_ERROR, "Invalid IOSurface Buffer");
-            goto fail;
         default:
             blog(LOG_ERROR, "Unexpected pixel format: %d (%c%c%c%c)", pixelFormat, pixelFormat >> 24, pixelFormat >> 16,
                  pixelFormat >> 8, pixelFormat);