Add support for extensions and experimental Vulkan support

docs/using-the-compiler.md

@@ -13,7 +13,7 @@ When using `rolling-translate` Varjo will check the data flowing from stage to s
 
 Stages are made using the [make-stage](http://techsnuffle.com/varjo/varjo-reference.html#VARJO.API%3AMAKE-STAGE) function.
 
-Vari supports all of the kinds of GLSL stage. When calling make-stage the stage kind is passed one of the following keywords:
+Vari supports all of the kinds of GLSL stages. When calling make-stage the stage kind is passed one of the following keywords:
 
 - `:vertex`
 - `:tessellation-control`
@@ -293,3 +293,260 @@ Just with this we already have enough that we could go back to using `define-var
 ### A quick note of v-defstruct
 
 As with `v-defun` there is a `v-defstruct` macro which is still commonly used. I'm trying to move to more consistent naming as we prepare to leave beta and so I advise preferring `define-vari-struct` over `v-defstruct`. It gives you nicer highlighting when using slime and will stay more consistent with the rest of public api as I update them.
+
+## Extensions
+
+We allow you to enable (or disable) GLSL extensions by passing them to Varjo as a sublist of the `context` argument of `make-stage`.
+While all keywords in the `context` list are interpreted as version specifiers, the last list whose first element is `:extensions` is used to enable or disable extensions.
+
+To enable extensions you can simply add the name of the extension as a string like this:
+
+    '(:400 (:extensions "GL_ARB_separate_shader_objects"))
+
+To specifiy a specific behavior for an extension you can use a list containing the name of the extension as its first element and the behavior as a keyword as the second element:
+
+    '(:400 (:extensions ("GL_ARB_shading_language_420pack" :require)))
+
+Allowed behaviors are `:enable`, `:disable`, `:warn` and `:require`.
+
+This is how extensions look when used in a stage:
+
+    TESTS> (glsl-code
+            (translate
+             (make-stage :fragment '((color :vec4))
+                         nil
+                         '(:400
+                           (:extensions
+                            "GL_ARB_separate_shader_objects"
+                            ("GL_ARB_shading_language_420pack" :require)))  
+                         '(color))))
+
+    "// fragment-stage
+    #version 400
+
+    #extension GL_ARB_separate_shader_objects : enable
+    #extension GL_ARB_shading_language_420pack : require
+
+    in _IN_BLOCK_
+    {
+         in vec4 COLOR;
+    } v_in;
+
+    layout(location = 0)  out vec4 _FRAGMENT_STAGE_OUT_0;
+
+    void main()
+    {
+        _FRAGMENT_STAGE_OUT_0 = v_in.COLOR;
+        return;
+    }
+
+    "
+
+## Experimental Vulkan Support
+
+Varjo offers experimental support for Vulkan (or SPIR-V).
+Currently only new qualifiers and types added for Vulkan are supported, but new GLSL functions, variables and shader stages (ray tracing and mesh shaders) are still missing.
+
+To enable Vulkan specific features you need to pass the target environment `:vulkan` to `make-stage` within the `context` argument.
+E.g. like this:
+
+    '(:440 (:target-environment :vulkan))
+
+This will add `location` qualifiers to members of interface blocks (e.g. `v_out`), since SPIR-V matches in/out variable only by their `location` and not their names.
+Because GLSL versions < 4.40 don't support `location` qualifiers for interface blocks, you must specifiy at least version `:440` when targetting Vulkan.
+E.g.:
+
+    TESTS> (glsl-code
+            (translate
+             (make-stage :fragment '((color :vec4))
+                         nil
+                         '(:440
+                           (:target-environment :vulkan))  
+                         '(color))))
+    "// fragment-stage
+    #version 440
+
+    in _IN_BLOCK_
+    {
+        layout(location = 0)  in vec4 COLOR;
+    } v_in;
+
+    layout(location = 0)  out vec4 _FRAGMENT_STAGE_OUT_0;
+
+    void main()
+    {
+        _FRAGMENT_STAGE_OUT_0 = v_in.COLOR;
+        return;
+    }
+    "
+
+### Descriptor Sets
+
+To specify the descriptor set and binding (within the set) for a uniform, you can pass them as extra qualifiers (`:set` and `:binding`) to the uniform definition.
+Since both qualifiers require a value the qualifiers must be specified as lists where the first element is the qualifier and the second one is the value.
+E.g.:
+
+    '((some-buffer my-uniform-buffer :ubo :std-140 (:set 0) (:binding 0)))
+
+Used in a vertex shader:
+
+    TESTS> (define-vari-struct uniform-buffer ()
+             (mvpc :mat4))
+    UNIFORM-BUFFER
+    TESTS> (glsl-code
+            (translate
+             (make-stage :vertex '((pos :vec4)
+                                   (color :vec4))
+                         '((uniform-buffer uniform-buffer :ubo :std-140 (:set 0) (:binding 0)))
+                         '(:440
+                           (:target-environment :vulkan))
+                         '((with-slots (mvpc) uniform-buffer
+                             (values
+                              (* mvpc pos)
+                              color))))))
+    "// vertex-stage
+    #version 440
+
+    layout(location = 0)  in vec4 POS;
+    layout(location = 1)  in vec4 COLOR;
+
+    out _FROM_VERTEX_STAGE_
+    {
+        layout(location = 0)  out vec4 _VERTEX_STAGE_OUT_1;
+    } v_out;
+
+    layout(std140, set = 0, binding = 0) uniform _UBO_UNIFORM_BUFFER
+    {
+        mat4 MVPC;
+    } UNIFORM_BUFFER;
+
+    void main()
+    {
+        vec4 g_PROG1_TMP1531 = (UNIFORM_BUFFER.MVPC * POS);
+        v_out._VERTEX_STAGE_OUT_1 = COLOR;
+        vec4 g_GEXPR0_1532 = g_PROG1_TMP1531;
+        gl_Position = g_GEXPR0_1532;
+        return;
+    }
+    "
+
+### Push Constants
+
+UBOs can be specified specified as push constants using the `:push-constant` qualifier.
+Like for other UBOs the default layout for push constants is `:std-140`:
+
+    TESTS> (glsl-code
+            (translate
+             (make-stage :vertex '((pos :vec4)
+                                   (color :vec4))
+                         '((uniform-buffer uniform-buffer :ubo :push-constant))
+                         '(:440
+                           (:target-environment :vulkan))
+                         '((with-slots (mvpc) uniform-buffer
+                             (values
+                              (* mvpc pos)
+                              color))))))
+    "// vertex-stage
+    #version 440
+
+    layout(location = 0)  in vec4 POS;
+    layout(location = 1)  in vec4 COLOR;
+
+    out _FROM_VERTEX_STAGE_
+    {
+        layout(location = 0)  out vec4 _VERTEX_STAGE_OUT_1;
+    } v_out;
+
+    layout(std140, push_constant) uniform _UBO_UNIFORM_BUFFER
+    {
+        mat4 MVPC;
+    } UNIFORM_BUFFER;
+
+    void main()
+    {
+        vec4 g_PROG1_TMP1556 = (UNIFORM_BUFFER.MVPC * POS);
+        v_out._VERTEX_STAGE_OUT_1 = COLOR;
+        vec4 g_GEXPR0_1557 = g_PROG1_TMP1556;
+        gl_Position = g_GEXPR0_1557;
+        return;
+    }
+    "
+
+### Textures & Samplers
+
+Vulkan allows splitting textures and samplers by introducting the new sampler types `:sampler` and `:sampler-shadows` which can be used to sample different texture types (e.g. `:itexture-2d`).
+
+However, the [glsl-spec](https://github.com/cbaggers/glsl-spec) doesn't yet specify the functions needed to construct a texture sampler from a sampler and a texture (e.g. `sampler2D(texture, sampler)`, so you'll have to use combined texture samplers (e.g. `:sampler-2d`) for now.
+
+### Subpass Inputs
+
+Subpass inputs require `input_attachment_index` to be set. You can specify this qualifier like `set` or `binding`:
+
+    '((albedo :subpass-input (:input-attachment-index 1) (:set 0) (:binding 1))
+
+Since the GLSL functions for reading subpass inputs (i.e. `subpassLoad`) are not yet specified in [glsl-spec](https://github.com/cbaggers/glsl-spec), they are pretty useless at the moment.
+
+### Vulkan Types
+
+The following new types are allowed when targetting Vulkan:
+
+- `:sampler`
+- `:sampler-shadow`
+- `:subpass-input`
+- `:subpass-input-ms`
+- `:isubpass-input`
+- `:isubpass-input-ms`
+- `:usubpass-input`
+- `:usubpass-input-ms`
+- `:texture-1d`
+- `:texture-1d-array`
+- `:texture-2d`
+- `:texture-2d-array`
+- `:texture-2d-ms`
+- `:texture-2d-ms-array`
+- `:texture-2d-rect`
+- `:texture-3d`
+- `:texture-buffer`
+- `:texture-cube`
+- `:texture-cube-array`
+- `:itexture-1d`
+- `:itexture-1d-array`
+- `:itexture-2d`
+- `:itexture-2d-array`
+- `:itexture-2d-ms`
+- `:itexture-2d-ms-array`
+- `:itexture-2d-rect`
+- `:itexture-3d`
+- `:itexture-buffer`
+- `:itexture-cube`
+- `:itexture-cube-array`
+- `:utexture-1d`
+- `:utexture-1d-array`
+- `:utexture-2d`
+- `:utexture-2d-array`
+- `:utexture-2d-ms`
+- `:utexture-2d-ms-array`
+- `:utexture-2d-rect`
+- `:utexture-3d`
+- `:utexture-buffer`
+- `:utexture-cube`
+- `:utexture-cube-array`
+- `:acceleration-structure-ext` (only if `GL_EXT_ray_query` is enabled)
+- `:ray-query-ext` (only if `GL_EXT_ray_query` is enabled)
+
+#### Initializing rayQueryEXT instances (GL_EXT_ray_query extension)
+
+Unlike most types in GLSL, `rayQueryEXT` instances cannot by constructed by calling a GLSL function.
+They are usually declared only in a function's scope and initialized by a call to `rayQueryInitializeEXT`:
+
+    void main() {
+        rayQueryEXT q;
+        rayQueryInitializeEXT(q, ...);
+        ...
+    }
+
+To achieve the same in Vari, we define an uninitialized typed symbol (e.g. using `let`) and the proceed as we normally would:
+
+    (let (((q :ray-query-ext)))
+      (ray-query-initialize-ext q ...))
+

package.lisp

@@ -232,7 +232,59 @@
            :v-uimage-3d
            :v-uimage-cube
            :v-uimage-cube-array
-           :v-uimage-rect))
+           :v-uimage-rect
+           ;;
+           :v-sampler-vulkan
+           :v-sampler-shadow-vulkan
+           ;;
+           :v-subpass-input
+           :v-subpass-input-ms
+           :v-isubpass-input
+           :v-isubpass-input-ms
+           :v-usubpass-input
+           :v-usubpass-input-ms
+           ;;
+           :v-itexture-1d
+           :v-itexture-1d-array
+           :v-itexture-2d
+           :v-itexture-2d-array
+           :v-itexture-2d-ms
+           :v-itexture-2d-ms-array
+           :v-itexture-2d-rect
+           :v-itexture-3d
+           :v-itexture-buffer
+           :v-itexture-cube
+           :v-itexture-cube-array
+           :v-texture-1d
+           :v-texture-1d-array
+           :v-texture-2d
+           :v-texture-2d-array
+           :v-texture-2d-ms
+           :v-texture-2d-ms-array
+           :v-texture-2d-rect
+           :v-texture-3d
+           :v-texture-buffer
+           :v-texture-cube
+           :v-texture-cube-array
+           :v-utexture-1d
+           :v-utexture-1d-array
+           :v-utexture-2d
+           :v-utexture-2d-array
+           :v-utexture-2d-ms
+           :v-utexture-2d-ms-array
+           :v-utexture-2d-rect
+           :v-utexture-3d
+           :v-utexture-buffer
+           :v-utexture-cube
+           :v-utexture-cube-array
+           :v-utexture-rect
+           :v-itexture-rect
+           :v-texture-rect
+           ;;
+           :v-acceleration-structure-ext
+           ;;
+           :v-ray-query-ext
+           ))
 
 ;;;; package.lisp
 (uiop:define-package #:varjo-conditions

src/docs.lisp

@@ -519,17 +519,53 @@ specified a struct type the following additional qualifiers are allowed:
 - `:std-140`
 - `:std-430`
 
+When targetting Vulkan the following additional qualifiers are allowed for uniforms:
+
+- `:set`
+- `:binding`
+- `:push-constant` (only for struct types)
+
+`:set` and `:binding` require an argument, so you have to specify them using a list like so:
+
+    (:set 0)
+
+where the second element is the value you want to set.
+
 The result of `make-stage` is an instance of one of the subclasses of the
 `stage` type.
 
 ### context
 
 The context argument must be a list that may contain any number of symbols from
-*supported-versions*. Context is used to specify the GLSL version to compile the
-stage.
+*supported-versions* as well as other lists for other context specific settings.
+Context is used to specify the GLSL version to compile the stage.
+
+Apart from GLSL versions, context can be used for specifying extension behaviour using a list in the following format:
+
+   (:extension \"GL_ARB_some_extension\")
+
+or:
+
+  (:extension (\"GL_ARB_some_extension1\" :enable) \"GL_ARB_some_extension2\")
+
+So either extensions are enabled by just passing their names as strings or a specific behaviour can be set for an extension using one of the following behaviours:
+
+- `:enable`
+- `:disable`
+- `:require`
+- `:warn`
+
+Furthermore a target environment can be specified using a list in the following format:
+
+  (:target-environment :opengl)
+
+Allowed target environments are
+
+- `:opengl`
+- `:vulkan`
 
-NOTE: The name 'context' is legacy at this point as it is only used to specify
-GLSL versions.
+The default is environment is `:opengl`.
+To enable Vulkan features you have to explicitly set `:vulkan` as the stage's target environment.
 
 ### code