Skip to content

Commit

Permalink
Allow customization of DXC executable
Browse files Browse the repository at this point in the history
Resolves o3de#17148 (DXC should be customizable via Settings Registry)
Which enables working around this bug:
o3de/o3de-extras#625

Developers can now pick a custom version of the DXC executable
via the Settings Registry Key:
`/O3DE/Atom/DxcOverridePath`.

Also, introduced the macro `PRE_HLSL_2021`, along with
an explanation in the following file:
/o3de/Gems/Atom/Feature/Common/Assets/ShaderLib/PRE_HLSL_2021.md
TL;DR: If a game project uses a DXC executable that doesn't conform
to the HLSL 2021 standard, then they should globally define the macro
PRE_HLSL_2021  when compiling shaders.

The O3DE default version of DXC is 1.7.2308 which conforms to HLSL 2021.
The problem is that some shaders get compiled to versions of SPIRV
not supported by Adreno GPU drivers. This commit gives the option
to use a different DXC and avoid this issue.

Removed redundant LinearToSRGB conversion functions.

Signed-off-by: galibzon <[email protected]>
  • Loading branch information
galibzon committed Nov 30, 2023
1 parent e5a4c94 commit ad9c676
Show file tree
Hide file tree
Showing 13 changed files with 185 additions and 97 deletions.
2 changes: 1 addition & 1 deletion Gems/Atom/Asset/Shader/Code/Source/Editor/AzslCompiler.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ namespace AZ
AZStd::string overridePath;
if (setReg->Get(overridePath, AzslCompilerOverridePath))
{
AZ_TraceOnce("AzslCompiler", "AZSLc executable override specified, using %s", azslcPath.c_str());
AZ_TraceOnce("AzslCompiler", "AZSLc executable override specified, using %s", overridePath.c_str());
azslcPath = AZStd::move(overridePath);
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,10 @@
real3 LinearSrgb_To_Srgb(real3 color)
{
// Copied from CryFx's ToAccurateSRGB()
#ifdef PRE_HLSL_2021
// See /o3de/Gems/Atom/Feature/Common/Assets/ShaderLib/PRE_HLSL_2021.md for details.
return (color.xyz < 0.0031308) ? 12.92 * color.xyz : 1.055 * pow(color.xyz, 1.0 / 2.4) - real3(0.055, 0.055, 0.055);
#else
return select((color.xyz < 0.0031308), 12.92 * color.xyz, 1.055 * pow(color.xyz, 1.0 / 2.4) - real3(0.055, 0.055, 0.055));
#endif
}
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,10 @@
real3 Srgb_To_LinearSrgb(real3 color)
{
// Copied from CryFx's ToAccurateLinear()
#ifdef PRE_HLSL_2021
// See /o3de/Gems/Atom/Feature/Common/Assets/ShaderLib/PRE_HLSL_2021.md for details.
return (color.xyz < 0.04045) ? color.xyz / 12.92h : pow((color.xyz + real3(0.055, 0.055, 0.055)) / real3(1.055, 1.055, 1.055), 2.4);
#else
return select((color.xyz < 0.04045), color.xyz / 12.92h, pow((color.xyz + real3(0.055, 0.055, 0.055)) / real3(1.055, 1.055, 1.055), 2.4));
#endif
}
Original file line number Diff line number Diff line change
Expand Up @@ -14,23 +14,23 @@ Academy Color Encoding System (ACES) software and tools are provided by the
non-exclusive right to copy, modify, create derivatives, and use, in source and
binary forms, is hereby granted, subject to acceptance of this license.

Copyright © 2015 Academy of Motion Picture Arts and Sciences (A.M.P.A.S.).
Copyright © 2015 Academy of Motion Picture Arts and Sciences (A.M.P.A.S.).
Portions contributed by others as indicated. All rights reserved.

Performance of any of the aforementioned acts indicates acceptance to be bound
by the following terms and conditions:

* Copies of source code, in whole or in part, must retain the above copyright
notice, this list of conditions and the Disclaimer of Warranty.
* Use in binary form must retain the above copyright notice, this list of
conditions and the Disclaimer of Warranty in the documentation and/or other
* Use in binary form must retain the above copyright notice, this list of
conditions and the Disclaimer of Warranty in the documentation and/or other
materials provided with the distribution.
* Nothing in this license shall be deemed to grant any rights to trademarks,
copyrights, patents, trade secrets or any other intellectual property of
* Nothing in this license shall be deemed to grant any rights to trademarks,
copyrights, patents, trade secrets or any other intellectual property of
A.M.P.A.S. or any contributors, except as expressly stated herein.
* Neither the name "A.M.P.A.S." nor the name of any other contributors to this
software may be used to endorse or promote products derivative of or based on
this software without express prior written permission of A.M.P.A.S. or the
this software without express prior written permission of A.M.P.A.S. or the
contributors, as appropriate.

This license shall be construed pursuant to the laws of the State of California,
Expand All @@ -40,19 +40,19 @@ and any disputes related thereto shall be subject to the jurisdiction of the
Disclaimer of Warranty: THIS SOFTWARE IS PROVIDED BY A.M.P.A.S. AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE,
AND NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL A.M.P.A.S., OR ANY
CONTRIBUTORS OR DISTRIBUTORS, BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, RESITUTIONARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
AND NON-INFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL A.M.P.A.S., OR ANY
CONTRIBUTORS OR DISTRIBUTORS, BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, RESITUTIONARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

////////////////////////////////////////////////////////////////////////////////
WITHOUT LIMITING THE GENERALITY OF THE FOREGOING, THE ACADEMY SPECIFICALLY
DISCLAIMS ANY REPRESENTATIONS OR WARRANTIES WHATSOEVER RELATED TO PATENT OR
OTHER INTELLECTUAL PROPERTY RIGHTS IN THE ACADEMY COLOR ENCODING SYSTEM, OR
WITHOUT LIMITING THE GENERALITY OF THE FOREGOING, THE ACADEMY SPECIFICALLY
DISCLAIMS ANY REPRESENTATIONS OR WARRANTIES WHATSOEVER RELATED TO PATENT OR
OTHER INTELLECTUAL PROPERTY RIGHTS IN THE ACADEMY COLOR ENCODING SYSTEM, OR
APPLICATIONS THEREOF, HELD BY PARTIES OTHER THAN A.M.P.A.S.,WHETHER DISCLOSED OR
UNDISCLOSED.
*/
Expand All @@ -62,7 +62,7 @@ UNDISCLOSED.
//
// You can find more detail about the ACES at the below link.
// https://www.oscars.org/science-technology/sci-tech-projects/aces
//
//
// You can find the ACES reference implementation repository at the below link.
// The reference implementation is implemented by the Color Transformation Language(CTL)
// which is defined by AMPAS. These implementations are find at the transforms directory.
Expand Down Expand Up @@ -103,7 +103,7 @@ struct SegmentedSplineParamsC5
float coefsLow[6]; // coefs for B-spline between minPoint and midPoint (units of log luminance)
float coefsHigh[6]; // coefs for B-spline between midPoint and maxPoint (units of log luminance)
float2 minPoint; // {luminance, luminance} linear extension below this
float2 midPoint; // {luminance, luminance}
float2 midPoint; // {luminance, luminance}
float2 maxPoint; // {luminance, luminance} linear extension above this
float slopeLow; // log-log slope of low linear extension
float slopeHigh; // log-log slope of high linear extension
Expand All @@ -113,7 +113,7 @@ struct SegmentedSplineParamsC9
{
float4 coefs[10]; // coefs for B-spline between minPoint and midPoint (units of log luminance)
float2 minPoint; // {luminance, luminance} linear extension below this
float2 midPoint; // {luminance, luminance}
float2 midPoint; // {luminance, luminance}
float2 maxPoint; // {luminance, luminance} linear extension above this
float slopeLow; // log-log slope of low linear extension
float slopeHigh; // log-log slope of high linear extension
Expand Down Expand Up @@ -196,15 +196,15 @@ float RGBToYc(float3 rgb, float ycRadiusWeight = 1.75)
{
// Converts RGB to a luminance proxy, here called YC
// YC is ~ Y + K * Chroma
// Constant YC is a cone-shaped surface in RGB space, with the tip on the
// Constant YC is a cone-shaped surface in RGB space, with the tip on the
// neutral axis, towards white.
// YC is normalized: RGB 1 1 1 maps to YC = 1
//
// ycRadiusWeight defaults to 1.75, although can be overridden in function
// ycRadiusWeight defaults to 1.75, although can be overridden in function
// call to RGBToYc
// ycRadiusWeight = 1 -> YC for pure cyan, magenta, yellow == YC for neutral
// ycRadiusWeight = 1 -> YC for pure cyan, magenta, yellow == YC for neutral
// of same value
// ycRadiusWeight = 2 -> YC for pure red, green, blue == YC for neutral of
// ycRadiusWeight = 2 -> YC for pure red, green, blue == YC for neutral of
// same value.

float r = rgb[0];
Expand Down Expand Up @@ -336,7 +336,7 @@ static const float pq_C = 10000.0;

// Converts from the non-linear perceptually quantized space to linear cd/m^2
// Note that this is in float, and assumes normalization from 0 - 1
// (0 - pq_C for linear) and does not handle the integer coding in the Annex
// (0 - pq_C for linear) and does not handle the integer coding in the Annex
// sections of SMPTE ST 2084-2014
float PerceptualQuantizerFwd(float N)
{
Expand All @@ -353,7 +353,7 @@ float PerceptualQuantizerFwd(float N)

// Converts from linear cd/m^2 to the non-linear perceptually quantized space
// Note that this is in float, and assumes normalization from 0 - 1
// (0 - pq_C for linear) and does not handle the integer coding in the Annex
// (0 - pq_C for linear) and does not handle the integer coding in the Annex
// sections of SMPTE ST 2084-2014
float PerceptualQuantizerRev(float C)
{
Expand Down Expand Up @@ -397,7 +397,7 @@ float Exp10(float f)
}

// Textbook monomial to basis-function conversion matrix.
static const float3x3 M =
static const float3x3 M =
{
{ 0.5, -1.0, 0.5 },
{ -1.0, 1.0, 0.5 },
Expand All @@ -411,7 +411,7 @@ static const SegmentedSplineParamsC5 RRT_PARAMS =
// coefsHigh[6]
{ -0.7185482425, 2.0810307172, 3.6681241237, 4.0000000000, 4.0000000000, 4.0000000000 },
{ 0.18*pow(2., -15), 0.0001 }, // minPoint
{ 0.18, 4.8 }, // midPoint
{ 0.18, 4.8 }, // midPoint
{ 0.18*pow(2., 18), 10000. }, // maxPoint
0.0, // slopeLow
0.0 // slopeHigh
Expand Down Expand Up @@ -494,7 +494,7 @@ float SegmentedSplineC9Fwd(float x, SegmentedSplineParamsC9 C)
float t = knot_coord - j;

float3 cf = { C.coefs[j].x, C.coefs[j + 1].x, C.coefs[j + 2].x };

float3 monomials = { t * t, t, 1. };
logy = dot(monomials, mul(cf, M));
}
Expand All @@ -510,7 +510,7 @@ float SegmentedSplineC9Fwd(float x, SegmentedSplineParamsC9 C)
logy = dot(monomials, mul(cf, M));
}
else
{
{
logy = logx * C.slopeHigh + (log10(C.maxPoint.y) - C.slopeHigh * log10(C.maxPoint.x));
}

Expand Down Expand Up @@ -689,7 +689,7 @@ float3 ReferenceRenderingTransform(float3 rgbIn)
////////////////////////////////////////////////////////////////////////////////
// Below implementation ported from ACES_parameterized.hlsl on the NVIDIA's HDR sample.
// The ACES implementation by ctl.
// https://github.com/ampas/aces-dev/blob/master/transforms/ctl/odt/sRGB/ODT.Academy.sRGB_100nits_dim.ctl
// https://github.com/ampas/aces-dev/blob/master/transforms/ctl/odt/sRGB/ODT.Academy.sRGB_100nits_dim.ctl

static const int APPLY_ALTER_SURROUND = 0x1;
static const int APPLY_DESATURATION = 0x2;
Expand All @@ -704,9 +704,9 @@ struct OutputTransformParameters
int outputDisplayTransformFlags; // Bit flag for control the ODT shader behavior
int outputDisplayTransformMode; // The ODT output mode
float2 cinemaLimits; // Reference white and black luminance values
float surroundGamma; // Gamma adjustment to be applied to compensate
float surroundGamma; // Gamma adjustment to be applied to compensate
// for the condition of the viewing environment.
// Note that ACES uses a value of 0.9811 for
// Note that ACES uses a value of 0.9811 for
// adjusting from dark to dim surrounding.
float gamma; // Optional gamma value that is applied as basic gamma curve OETF
SegmentedSplineParamsC9 acesSplineParams; // ACES spline parameters
Expand Down Expand Up @@ -764,9 +764,9 @@ float3 OutputDeviceTransform(const float3 oces, OutputTransformParameters otPara
float3 outputCV = linearCV;

if (otParams.outputDisplayTransformMode == OUTPUT_SRGB)
{
{
// SRGB
// clamp 0/1 and encode
// clamp 0/1 and encode
linearCV = clamp(linearCV, 0., 1.);

outputCV[0] = MoncurveR(linearCV[0], DISPGAMMA, OFFSET);
Expand All @@ -789,8 +789,12 @@ float3 OutputDeviceTransform(const float3 oces, OutputTransformParameters otPara
{
// LDR mode, clamp 0/1 and encode with given gamma value for the OETF
linearCV = clamp(linearCV, 0., 1.);

#ifdef PRE_HLSL_2021
// See /o3de/Gems/Atom/Feature/Common/Assets/ShaderLib/PRE_HLSL_2021.md for details.
outputCV = outputCV > 0.0f ? pow(linearCV, 1.0f / otParams.gamma) : 0.0f;
#else
outputCV = select(outputCV > 0.0f, pow(linearCV, 1.0f / otParams.gamma), 0.0f);
#endif
}

return outputCV;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,41 +8,9 @@

#pragma once

/*
Linear to sRGB and back again
The following 4 functions are helpful for accurately converting from
Linear colorspace to sRGB colorspace and back again. With both floats and halfs.

More information, including the transformations used in this file, can be found here:
https://en.wikipedia.org/wiki/SRGB
*/

float3 LinearToSRGB(float3 col)
{
return select((col.xyz < 0.0031308), 12.92 * col.xyz, 1.055 * pow(abs(col.xyz), 1.0 / 2.4) - float3( 0.055, 0.055, 0.055 )) ;
}

float3 SRGBToLinear(float3 col)
{
return select((col.xyz < 0.04045), col.xyz / 12.92, pow (abs((col.xyz + float3(0.055, 0.055, 0.055)) / float3(1.055, 1.055, 1.055)), 2.4));
}

/* AZSL doesn't support function overload for the moment.
half3 LinearToSRGB(half3 col)
{
return select((col < 0.0031308h), 12.92h * col, 1.055h * pow(abs(col), 1.0h / 2.4h) - half3(0.055h, 0.055h, 0.055h));
}

half3 SRGBToLinear(half3 col)
{
return select((col.xyz < 0.04045h), col.xyz / 12.92h, pow (abs((col.xyz + half3(0.055h, 0.055h, 0.055h)) / half3(1.055h, 1.055h, 1.055h)), 2.4h));
}
*/

// Lifted from Legacy LY Renderer
// Input color must be in linear space
float GetLuminance( float3 color )
{
return dot( color, float3( 0.2126f, 0.7152f, 0.0722f ) );
}

75 changes: 75 additions & 0 deletions Gems/Atom/Feature/Common/Assets/ShaderLib/PRE_HLSL_2021.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# About the macro PRE_HLSL_2021

The macro PRE_HLSL_2021 was introduced since this bug: https://github.com/o3de/o3de-extras/issues/625.

The workaround to the bug mentioned above was to use an older version of DXC 1.6.2112, instead of DXC 1.7.2308. DXC 1.7.2308 conforms to HLSL 2021 which doesn't support the ternary conditional operator (`(cond) ? true_statement : false_statement`) for vectors. HLSL 2021 only supports ternary conditional operator for scalars. Starting HLSL 2021, to achieve an equivalent behavior for ternary conditional operator for vectors, the `select()` intrinsic was introduced. See more details here:
https://devblogs.microsoft.com/directx/announcing-hlsl-2021/

This is why you'll find now code like this in some *.azsli or *.azsl files:
```hlsl
#ifdef PRE_HLSL_2021
return (color.xyz < 0.04045) ? color.xyz / 12.92h : pow((color.xyz + real3(0.055, 0.055, 0.055)) / real3(1.055, 1.055, 1.055), 2.4);
#else
return select((color.xyz < 0.04045), color.xyz / 12.92h, pow((color.xyz + real3(0.055, 0.055, 0.055)) / real3(1.055, 1.055, 1.055), 2.4));
#endif
```

By default the macro `PRE_HLSL_2021` is not defined, and the shaders will use the `and`, `or` and `select` intrinsics.

## When should the developer define the PRE_HLSL_2021 macro?
Defining the PRE_HLSL_2021 is required if the developer customizes the DXC executable to version pre-HLSL 2021 like DXC 1.6.2112.

The DXC executable can be customized via the Settings Registry Key:
`/O3DE/Atom/DxcOverridePath`.
Here is a customization example in a *.setreg file:
```json
{
"O3DE": {
"Atom": {
"DxcOverridePath": "C:\\Users\\galib\\.o3de\\3rdParty\\packages\\DirectXShaderCompilerDxc-1.6.2112-o3de-rev1-windows\\DirectXShaderCompilerDxc\\bin\\Release\\dxc.exe"
}
}
}
```

## How to globally define the PRE_HLSL_2021 macro when compiling all shaders?
Customization of command line arguments for shader compilation tools is described here:
https://github.com/o3de/o3de/wiki/%5BAtom%5D-Developer-Guide:-Shader-Build-Arguments-Customization

In particular if you don't want to mess with engine files, in your game project:
1- Define an alternate location for `shader_build_options.json` using the registry key: `/O3DE/Atom/Shaders/Build/ConfigPath`.
2- Your new `shader_build_options.json` should be a copy of engine original located at `C:\GIT\o3de\Gems\Atom\Asset\Shader\Config\shader_build_options.json` with the addition of:
```json
"Definitions": [ "PRE_HLSL_2021" ], // Needed when using DXC 1.6.2112
```
Complete example of customized shader_build_options.json:
```json
{
"Definitions": [ "PRE_HLSL_2021" ], // Needed when using DXC 1.6.2112
"AddBuildArguments": {
"debug": false,
"preprocessor": ["-C" // Preserve comments
, "-+" // C++ mode
],
"azslc": ["--full" // Always generate the *.json files with SRG and reflection info.
, "--Zpr" // Row major matrices.
, "--W1" // Warning Level 1
, "--strip-unused-srgs" // Remove unreferenced SRGs.
, "--root-const=128"
],
"dxc": ["-Zpr" // Row major matrices.
],
// The following apply for all Metal based platforms.
"spirv-cross": ["--msl"
, "--msl-version"
, "20100"
, "--msl-invariant-float-math"
, "--msl-argument-buffers"
, "--msl-decoration-binding"
, "--msl-texture-buffer-native"
]
}
}
```

Finally, if not done already you'd have to restart the AssetProcessor so it picks the changes from the Settings Registry.
Loading

0 comments on commit ad9c676

Please sign in to comment.