Bladeren bron

libobs: Fix conversion shader D3D display bug

Just for a quick background: D3D's fmod intrinsic is very imprecise.
Naturally floating points aren't precise at all, and when the numbers
you're dealing with become very large, it can often be off by 0.1 or
more.

However, apparently 0.1 isn't enough of an offset to ensure a proper
value when using the fmod intrinsic and then flooring the value.  0.2
seems to fix the issue and make the image display properly.
jp9000 11 jaren geleden
bovenliggende
commit
ca8a9fb5a7
1 gewijzigde bestanden met toevoegingen van 1 en 1 verwijderingen
  1. 1 1
      libobs/data/format_conversion.effect

+ 1 - 1
libobs/data/format_conversion.effect

@@ -59,7 +59,7 @@ VertInOut VSDefault(VertInOut vert_in)
 }
 
 /* used to prevent internal GPU precision issues width fmod in particular */
-#define PRECISION_OFFSET 0.1
+#define PRECISION_OFFSET 0.2
 
 float4 PSNV12(VertInOut vert_in) : TARGET
 {