diff --git a/docs/reference-manual/devices/library/window-device.md b/docs/reference-manual/devices/library/window-device.md index 753c7ecf..1cbcba38 100644 --- a/docs/reference-manual/devices/library/window-device.md +++ b/docs/reference-manual/devices/library/window-device.md @@ -50,9 +50,24 @@ The following properties describe the single-window device: ## Mouse input -* `/cursor/scaled`: mouse cursor position in the window scaled to `[0;1]` bounds. +* `/cursor/scaled`: mouse cursor position in the window scaled to `[0;1]` bounds, y down. +* `/cursor/gl`: the same with y up, as OpenGL texture coordinates. +* `/cursor/clip`: mouse cursor position in clip space, x and y in `[-1;1]`, y up: the `mouse` of [vertexshaderart.com](https://www.vertexshaderart.com). * `/cursor/absolute`: mouse cursor position in the window in pixels. +In **Multi-Window Mapping** mode, each window has these under its index prefix (`/0/cursor/clip`...). + +## Touch input + +In **Single Window** and **Background** modes: + +* `/touch/points`: the pointers, a list of 32 `[x, y, pressure]` in clip space, y up. Slot 0 is the mouse (pressure 1 while a button is pressed) or the first finger, then one slot per finger in the order they came down; a released slot keeps its position, with pressure 0. Writable: a value sent to it moves the pointers. +* `/touch/history`: a texture, the last 240 frames of `/touch/points`: 32 × 240 RGBA float, one column per pointer, the newest frame in the row at `v = 0`, each texel `(x, y, pressure, time)`. This is the `touch` texture of vertexshaderart.com; route it to a texture inlet as any texture address. + +## Shader inputs + +Shader inputs named for the pointers are bound to these addresses of the first window device by default, in ISF, Vertex Shader Art and Render Pipeline shaders alike: a `point2D` named `iMouse` to `/cursor/absolute`, a `mouse` of a Vertex Shader Art shader to `/cursor/clip`, another `point2D` whose name contains `mouse` to `/cursor/gl`, and an image named `touch` to `/touch/history`. The address shows on the inlet, where it can be changed. See [ISF pointer inputs]({{ site.baseurl }}/reference/languages/isf.html#pointer-inputs) and [VSA mouse and touch]({{ site.baseurl }}/reference/languages/vertex-shader-art.html#mouse-and-touch). + ## Tablet input * `/tablet/scaled`: tablet pen position in the window scaled to `[0;1]` bounds. diff --git a/docs/reference-manual/languages/csf.md b/docs/reference-manual/languages/csf.md index 0a3b9778..5e39448d 100644 --- a/docs/reference-manual/languages/csf.md +++ b/docs/reference-manual/languages/csf.md @@ -44,33 +44,36 @@ void main() { |---|---:|---| | `MODE` | yes | Must be `"COMPUTE_SHADER"`. | | `PASSES` | yes | Compute dispatches to run each frame. | -| `RESOURCES` | no | GPU resources, ports, and controls. Preferred CSF spelling. | -| `INPUTS` | no | Alias accepted by the parser; useful for shared ISF-style conventions. | +| `RESOURCES` | no | GPU resources, ports, and controls. | +| `INPUTS` | no | Deprecated in a compute shader, with a warning: its entries keep their ISF meaning, see below. | | `TYPES` | no | User-defined struct layouts used by buffers and geometry attributes. | | `DESCRIPTION`, `CREDIT`, `CATEGORIES`, `ISFVSN` | no | Metadata. | -`RESOURCES` and `INPUTS` are parsed with the same resource vocabulary. In CSF examples, use `RESOURCES` to make the compute intent clear. +The header follows the [ISF header syntax]({{ site.baseurl }}/reference/languages/isf.html#header-syntax) (case-independent keys, comments, error messages) and the [input name rules]({{ site.baseurl }}/reference/languages/isf.html#input-names), which apply to resource names. + +A compute shader declares its controls and resources in `RESOURCES`. An `INPUTS` list is still read, with the ISF meaning of its entries: an `image` there is a sampled texture unless it declares `ACCESS` or `FORMAT`, and `IS_DEPTH` is refused on it. The shader then gets the warning *INPUTS is deprecated in a COMPUTE_SHADER*: move the entries to `RESOURCES`, turning the sampled images into `texture` entries. ## Resource types -Each resource has `NAME` and `TYPE`. Type names are case-insensitive in practice, but lower-case matches most *score* examples. +Each resource has `NAME` and `TYPE`; a `RESOURCES` entry without a `TYPE`, or with one the parser does not know, is an error naming the entry. Type names are case-independent, but lower-case matches most *score* examples. | Type | Meaning | |---|---| -| `image` | Storage image when `ACCESS` or `FORMAT` is present; otherwise sampled texture input. | +| `image` | Storage image (`image2D`, `image3D`, `image2DArray`), read and written with `IMG_LOAD` / `IMG_STORE`. | +| `image_cube` | Writable cubemap storage image. | | `texture` | Read-only sampled texture. | | `cubemap` | Read-only sampled cubemap. | -| `storage` | Structured SSBO. If `TYPE` is omitted, storage is the default. | +| `storage` | Structured SSBO. | | `uniform` | std140 UBO. Read-only, small data. | | `geometry` | GPU geometry port with per-attribute buffers. | | `float`, `long`, `bool`, `event`, `point2D`, `point3D`, `color` | UI controls and uniforms. | -| `audio`, `audioFFT`, `audioHist` | Audio textures. | +| `audio`, `audioFFT`, `audioHistogram` | Audio textures. | `event` inputs accept an impulse as well as `true`, and fire for one rendered frame. Use an event for a simulation reset rather than a permanently enabled boolean. ## Images -A CSF `image` resource can be read, written, or read/write. +A CSF `image` resource is a storage image: it can be read, written, or read/write. To sample an image with a filter, declare a [`texture`](#sampled-textures) instead. ```json { @@ -86,15 +89,15 @@ A CSF `image` resource can be read, written, or read/write. | Field | Description | |---|---| | `ACCESS` | `read_only`, `write_only`, or `read_write`. | -| `FORMAT` | GLSL image format such as `rgba8`, `rgba16f`, `rgba32f`, `r32f`, `rgba32ui`. | -| `WIDTH`, `HEIGHT`, `DEPTH` | Size expressions. `DEPTH` creates a 3D image. | +| `FORMAT` | GLSL image format, see [Image formats](#image-formats). The image is allocated in that format. | +| `WIDTH`, `HEIGHT`, `DEPTH` | Size expressions. `DEPTH` creates a 3D image. Each of `WIDTH` and `HEIGHT` takes its expression, else the size of the first input image, else the render size. | | `DIMENSIONS` | `2` or `3`. Alternative way to declare 3D images. | -| `IS_ARRAY` / `ARRAY` | 2D texture array image. | +| `IS_ARRAY` / `ARRAY` | 2D texture array image. Not with a cubemap or a 3D image: both are refused. | | `LAYERS` | Array layer count expression. | -| `CUBEMAP` / `IS_CUBE` | Cubemap storage image. | -| `PERSISTENT` | Ping-pong image history; the previous frame is available as `_prev`. | +| `CUBEMAP` / `IS_CUBE` | Cubemap storage image, as `TYPE: "image_cube"`. | +| `PERSISTENT` | Image history; the previous frame is available as `_prev`. Requires `ACCESS: "read_write"`. | | `GENERATE_MIPS` | Generate mipmaps after the dispatch. Valid for 2D images. | -| `VISIBILITY` | Binding visibility when the resource is shared with graphics stages: `compute`, `fragment`, `vertex`, or `vertex+fragment`. | +| `VISIBILITY` | Shader stages that see the image: `compute` (the default in a compute shader), `fragment`, `vertex`, `vertex+fragment` (also spelled `both`, `graphics` or `all`) or `none`, in any case. Another value is an error. | | `COMPOSITE` | How the published image composites when consumed directly: `over`, `add`, `multiply`, `screen`, `replace`. | Use the portable image helpers when orientation matters: @@ -105,7 +108,7 @@ IMG_STORE(outputImage, ivec2(x, y), c); IMG_STORE_LAYER(arrayImage, ivec3(x, y, layer), c); ``` -Integer storage formats such as `rgba32ui` require Qt 6.10 or newer, plus device support. With earlier Qt versions, unsupported format names fall back to RGBA8; that fallback does not preserve integer shader semantics. See [[Graphics pipeline]] for sizing and backend restrictions. +Integer storage formats such as `rgba32ui` require Qt 6.10 or newer, plus device support. With earlier Qt versions, they are allocated as RGBA8; that fallback does not preserve integer shader semantics. See [[Graphics pipeline]] for sizing and backend restrictions. Raw `imageLoad` / `imageStore` also work, but they do not apply *score*'s coordinate-origin fixups. @@ -132,19 +135,29 @@ Dispatch with `EXECUTION_MODEL: { "TYPE": "3D_IMAGE", "TARGET": "volume" }` and "WIDTH": 1024, "HEIGHT": 1024, "PERSISTENT": true } ``` -`state` is the current writable image. `state_prev` is the previous frame's read-only image. `PERSISTENT` with `write_only` is invalid because there is no meaningful previous read path. +`state` is the current writable image. `state_prev` is a separate read-only image holding the previous frame. On the first frame, `state_prev` is filled from `state` before each pass, so a pass reads what the earlier passes of that frame wrote, never what it is writing itself. The image the node outputs is the one written this frame. `PERSISTENT` requires `read_write`: with `read_only` or `write_only` the header is refused. ## Sampled textures -A read-only sampled texture is declared with `TYPE: "texture"`, or with `TYPE: "image"` when no `ACCESS` / `FORMAT` is present. +A read-only sampled texture is declared with `TYPE: "texture"` (or `"cubemap"` for a cubemap). ```json { "NAME": "lookup", "TYPE": "texture", "FILTER": "nearest", "WRAP": "clamp_to_edge" } { "NAME": "volumeTex", "TYPE": "texture", "DIMENSIONS": 3 } +{ "NAME": "layers", "TYPE": "texture", "IS_ARRAY": true } { "NAME": "sky", "TYPE": "cubemap", "MIPMAP_MODE": "linear" } ``` -Sampler fields are the same as ISF: `WRAP`, `WRAP_S`, `WRAP_T`, `WRAP_R`, `FILTER`, `MIN_FILTER`, `MAG_FILTER`, `MIPMAP_MODE`, `BORDER_COLOR`, `COMPARE`, `ANISOTROPY`, `LOD_BIAS`, `MIN_LOD`, `MAX_LOD`. +| Field | Description | +|---|---| +| `DIMENSIONS` | `2` or `3` (`sampler3D`). | +| `IS_ARRAY` | A 2D texture array (`sampler2DArray`). | +| `CUBE` | A cubemap (`samplerCube`), without `DIMENSIONS: 3` or `IS_ARRAY`. | +| `STATIC` | Bind the producer's texture directly instead of rendering it into the input's own target. | + +`IS_DEPTH` is refused on a `texture`: the `_depth` companion only exists for an [ISF image input]({{ site.baseurl }}/reference/languages/isf.html#sampleable-depth). + +Sampler fields and their resolution are the same as in ISF: `WRAP`, `WRAP_S`, `WRAP_T`, `WRAP_R`, `FILTER`, `MIN_FILTER`, `MAG_FILTER`, `MIPMAP_MODE` and `COMPARE`, each field taken from the inlet when the user set it, else from the JSON, else the default. See [ISF sampler state]({{ site.baseurl }}/reference/languages/isf.html#sampler-state), which also lists the keys accepted without effect. ## Storage buffers @@ -168,11 +181,10 @@ Fields: | Field | Description | |---|---| | `ACCESS` | `read_only`, `write_only`, or `read_write`. | -| `LAYOUT` | Struct fields. Last field may be a flexible array with `[]`. | -| `SIZE` | Flexible array length when needed. | -| `PERSISTENT` | Ping-pong buffer history; previous frame is `_prev`. | +| `LAYOUT` | Struct fields, required. The last field may be a flexible array with `[]`. | +| `PERSISTENT` | Buffer history; previous frame is `_prev`. Requires `ACCESS: "read_write"`. | | `VISIBILITY` | Graphics-stage visibility when shared: `compute`, `fragment`, `vertex`, `vertex+fragment`, or `none`. | -| `BUFFER_USAGE` | Special usage: `indirect_draw` or `indirect_draw_indexed`. Allows empty `LAYOUT`. Indirect dispatch needs no special usage: point `EXECUTION_MODEL` `TYPE: "INDIRECT"` at an ordinary storage buffer (see [Indirect dispatch](#indirect-dispatch)). | +| `BUFFER_USAGE` | Special usage: `indirect_draw` or `indirect_draw_indexed`. Indirect dispatch needs no special usage: point `EXECUTION_MODEL` `TYPE: "INDIRECT"` at an ordinary storage buffer (see [Indirect dispatch](#indirect-dispatch)). | Access fields as members: @@ -181,6 +193,12 @@ uint n = particles.count; vec4 p = particles.pos[i]; ``` +A field `TYPE` may have several array extents, each a positive integer constant expression: `vec4[2][3]`, `float[4 * 8]`. Only the outermost one may be runtime-sized (`vec4[][3]`). A layout that cannot be sized this way is refused with a message naming the field. + +A writable buffer whose `LAYOUT` ends in a flexible array gets a **` size`** control (1 to 536870911, default 1024): the element count of that array. Changing it recreates the buffer, zero-filled; a `PERSISTENT` buffer and its `_prev` start again from zeros. A buffer is limited by what the device binds (Vulkan `maxStorageBufferRange`, OpenGL `GL_MAX_SHADER_STORAGE_BLOCK_SIZE`); one that cannot be allocated at its declared size stops the shader's passes, with a warning naming it, until its size is reduced. + +A read-only storage resource fed by a cable binds the range of the buffer its producer publishes, at the device's offset alignment, so `.length()` is the published element count. A buffer without storage usage, or published at an unaligned offset, is not bound: the resource keeps its own zero-filled buffer, and a warning is printed once. + ## Uniform buffers `uniform` declares a std140 UBO. Use it for small read-only data such as cameras, counts, and constants. @@ -197,6 +215,8 @@ vec4 p = particles.pos[i]; } ``` +A `uniform` resource gets an inlet. It binds the buffer cabled there, at the range its producer publishes, when that buffer has uniform usage and holds at least the block's fixed part plus one element of a trailing array. Otherwise the block reads a zero-filled buffer sized from its `LAYOUT`, and a warning is printed once. + For writable or large data, use `storage` instead. ## Geometry resources @@ -251,6 +271,10 @@ Set `TOPOLOGY` explicitly when producing points or lines. The topology travels w | `REQUIRED` | `false` allows missing upstream attributes to use a fallback. | | `COPY_FROM` | Forward an attribute from another geometry without shader writes. | +An attribute read from an upstream geometry is decoded from its source format (32-bit float, 8-bit normalized, half float, 32-bit signed or unsigned integer) into the type the shader declares, with the usual `(0, 0, 0, 1)` fill for missing components. A float read as `uint` or `int` is truncated and clamped to the type's range, NaN and negative values giving 0 for `uint`. Scalar and vector types convert into each other this way. + +An attribute the shader writes is published as what it is: `int` / `ivec*` with signed formats, `uint` / `uvec*` with unsigned ones. A `bool`, matrix or struct attribute is published as a user struct carrying its GLSL type name and std430 element size. A compute shader reads such an attribute only as the same type at the same size; otherwise it reads zeros, and one warning names both types. A raster vertex input matching it refuses its pipeline with a warning. + `ACCESS: "gather"` means read/write plus foreign-index reads. Use it for neighbour loops, N-body interactions, stencils, and any shader that reads indices other than its own invocation. It forces separate `_in` and `_out` buffers so reads see a stable snapshot. Access attributes with macros: @@ -281,9 +305,22 @@ The forwarded attribute is not bound for shader reads or writes. `INDIRECT` allocates an indirect-command buffer associated with the geometry. The shader fills draw commands and optionally a draw-count buffer. The runtime uses GPU indirect drawing when available and compatible fallbacks otherwise. -Use `ISF_INDIRECT(geo)` and, with `DRAW_COUNT`, `ISF_INDIRECT_COUNT(geo)` in GLSL. +Use `ISF_INDIRECT(geo)` and, with `DRAW_COUNT`, `ISF_INDIRECT_COUNT(geo)` in GLSL; both work for any number of `INDIRECT` geometries in one shader. The records are the GPU's own command layouts, five words each: + +```glsl +struct DrawIndirectCommand { // "INDEXED": false + uint vertexCount; uint instanceCount; uint firstVertex; uint firstInstance; + int baseVertex; // unused by a non-indexed draw +}; +struct DrawIndexedIndirectCommand { // "INDEXED": true + uint indexCount; uint instanceCount; uint firstIndex; int baseVertex; + uint firstInstance; +}; +``` + +With `DRAW_COUNT`, the commands at or past the count the shader writes are not drawn, whatever they hold, on every backend: where the device has no draw-count support, a compute pass copies the commands with those slots zeroed before drawing, without reading anything back. -When allocating a capacity of commands, initialize inactive slots to zero and guard indirectly dispatched work against the live count. Some backends use bounded dispatch or draw fallbacks rather than native GPU-count execution. [[Shader cookbook]] shows the corresponding corpus recipe, plus geometry forwarding and audio input examples. +When allocating a capacity of commands, guard indirectly dispatched work against the live count. Some backends use bounded dispatch or draw fallbacks rather than native GPU-count execution. [[Shader cookbook]] shows the corresponding corpus recipe, plus geometry forwarding and audio input examples. ## Auxiliary buffers and textures @@ -305,6 +342,8 @@ Geometry can carry extra data besides attributes. ] ``` +A `read_write` auxiliary is one buffer that keeps its contents from frame to frame, also when the geometry is fed back into the same shader: `_in`, `_out` and the plain name all refer to it. For a previous-frame copy, use a `PERSISTENT` storage buffer (`_prev`), an attribute with `ACCESS: "gather"`, or two buffers alternated on even and odd frames. + ### UBO auxiliary ```json @@ -324,12 +363,14 @@ Auxiliary textures travel with the geometry and do not create score input ports. ```json "AUXILIARY": [ { "NAME": "base_color", "TYPE": "texture", "FILTER": "linear" }, - { "NAME": "shadow_atlas", "TYPE": "image", "DEPTH": true, "COMPARE": "less" }, + { "NAME": "shadow_atlas", "TYPE": "image", "IS_DEPTH": true, "COMPARE": "less" }, { "NAME": "history", "TYPE": "storage_image", "FORMAT": "rgba16f", "ACCESS": "read_write", "WIDTH": 1024, "HEIGHT": 1024 } ] ``` -Auxiliary texture `TYPE` values include `image`, `texture`, `cubemap`, `image_cube`, `storage_image`, `storage_cube`, `storage_image_array`, and `storage_3d`. +Auxiliary texture `TYPE` values include `image`, `texture`, `cubemap`, `image_cube`, `storage_image`, `storage_cube`, `storage_image_array`, and `storage_3d`. An auxiliary texture is a storage image only when its `TYPE` is one of the `storage_*` ones; a `FORMAT` alone does not make it one, and a `"STORAGE"` key is refused. + +`IS_DEPTH: true` marks a texture that holds depth, sampled with `COMPARE` for shadows; it binds that one texture, with no `_depth` companion. `DEPTH` is the depth of a volume, a number or an expression. ## Custom types @@ -359,10 +400,12 @@ CSF shaders run one or more dispatch passes per frame. | Pass field | Description | |---|---| -| `LOCAL_SIZE` | Workgroup size `[x, y, z]`. Defaults to `[16, 16, 1]`. | +| `LOCAL_SIZE` | Workgroup size `[x, y, z]`, three integers of at least 1. Defaults to `[16, 16, 1]`. | | `EXECUTION_MODEL` | Dispatch strategy. | -`PASSINDEX` is the current pass number. +Every `PASSES` entry of a compute shader is a dispatch, with or without `LOCAL_SIZE`, run in the order written. `PASSINDEX` is the current pass number. + +Workgroup counts are computed in 64 bits against the device's per-dimension limit. A pass that needs more workgroups than the device dispatches is skipped, with a warning. ### Execution model types @@ -372,7 +415,7 @@ CSF shaders run one or more dispatch passes per frame. | `3D_IMAGE` | Width, height, depth of target image | `TARGET`, optional `STRIDE_X/Y/Z` | | `PER_VERTEX` | Vertex count of a geometry | optional strides | | `PER_INSTANCE` | Instance count of a geometry | optional strides | -| `1D_BUFFER` | Elements of a storage buffer | `TARGET` | +| `1D_BUFFER` | Elements of a storage buffer: the element count of its trailing array, `$COUNT_` | `TARGET`; without one, the storage buffer the shader outputs, or with none, the vertices of the first geometry | | `MANUAL` | Fixed workgroup count | `WORKGROUPS` | | `USER` | User-controlled workgroup count | creates X/Y/Z dispatch controls | | `INDIRECT` | Workgroups read from a storage buffer | `TARGET`, `WORKGROUPS` fallback ceiling, optional `OFFSET` | @@ -393,7 +436,7 @@ CSF shaders run one or more dispatch passes per frame. } ``` -`TARGET` is a storage buffer containing a `{x, y, z}` `uint` triplet. `WORKGROUPS` is the fallback ceiling used when the backend cannot dispatch indirectly; the shader must still bounds-check. +`TARGET` is a storage buffer containing a `{x, y, z}` `uint` triplet at byte `OFFSET`, relative to the part of the buffer the shader sees. `WORKGROUPS` is the fallback ceiling used when the backend cannot dispatch indirectly; the shader must still bounds-check. When `OFFSET` is not a multiple of 4, or the 12 bytes of the triplet do not fit in the buffer from there, the pass runs with its `WORKGROUPS` ceiling and a warning is printed once. ## Expressions @@ -406,8 +449,12 @@ Many fields accept expressions: | `$WIDTH_name`, `$HEIGHT_name`, `$DEPTH_name`, `$LAYERS_name` | Texture dimensions. | | `$VERTEX_COUNT`, `$INSTANCE_COUNT` | Counts of the first geometry. | | `$VERTEX_COUNT_geo`, `$INSTANCE_COUNT_geo` | Counts of a named geometry. | +| `$COUNT_buf` | Element count of the trailing array of a storage or uniform buffer: the elements its allocation holds for a flexible array, the product of the extents for a fixed one, 1 when the layout ends in no array. A capacity, not the number of elements a shader has written. | +| `$BYTESIZE_buf` | Byte size of that buffer's allocation. | + +`$COUNT_` and `$BYTESIZE_` cover the buffers the shader declares and those carried by its input geometries. When both have the same name, the one the shader declares at the top level is used. -Supported operators include `+`, `-`, `*`, `/`, `%`, comparisons, `&&`, and `||`. Supported functions include `abs`, `sqrt`, `sin`, `cos`, `tan`, `exp`, `log`, `ceil`, `floor`, `round`, `min`, `max`, `pow`, `clamp`, `lerp`, `step`, and `smoothstep`. +Supported operators include `+`, `-`, `*`, `/`, `%`, comparisons, `&&`, and `||`. Supported functions include `abs`, `sqrt`, `sin`, `cos`, `tan`, `exp`, `log`, `ceil`, `floor`, `round`, `min`, `max`, `pow`, `clamp`, `lerp`, `step`, and `smoothstep`. `lerp`, `step` and `smoothstep` take their arguments in another order than in GLSL: see [ISF size expressions]({{ site.baseurl }}/reference/languages/isf.html#size-expressions). ## Built-in uniforms @@ -421,19 +468,19 @@ Supported operators include `+`, `-`, `*`, `/`, `%`, comparisons, `&&`, and `||` | `RENDERSIZE` | `vec2` | Render size in pixels. | | `DATE` | `vec4` | `(year, month, day, seconds)`. | | `SAMPLERATE` | `float` | Audio sample rate. | -| `gl_NumWorkGroups` / `isf_NumWorkGroups` | `uvec3` | Actual workgroup count for the current dispatch. | +| `gl_NumWorkGroups` / `isf_NumWorkGroups` | `uvec3` | Workgroup count of the current dispatch. On Direct3D it is a uniform: in an `INDIRECT` pass it holds the `WORKGROUPS` ceiling, not the count the GPU dispatches, and a warning says so. | ## Image formats -Common image formats: +The storage image formats are the ones the engine can allocate: | Category | Formats | |---|---| -| Float / normalized | `rgba32f`, `rgba16f`, `rgba8`, `rg32f`, `rg16f`, `rg8`, `r32f`, `r16f`, `r8`, `rgb10_a2` | -| Signed integer | `rgba32i`, `rgba16i`, `rgba8i`, `rg32i`, `rg16i`, `rg8i`, `r32i`, `r16i`, `r8i` | -| Unsigned integer | `rgba32ui`, `rgba16ui`, `rgba8ui`, `rg32ui`, `rg16ui`, `rg8ui`, `r32ui`, `r16ui`, `r8ui` | +| Float / normalized | `rgba32f`, `rgba16f`, `rgba8`, `rg8`, `rg16`, `r32f`, `r16f`, `r16`, `r8`, `rgb10_a2` | +| Signed integer (Qt 6.10+) | `rgba32i`, `rg32i`, `r32i`, `r8i` | +| Unsigned integer (Qt 6.10+) | `rgba32ui`, `rg32ui`, `r32ui`, `r8ui` | -Format names are case-insensitive, but lower-case GLSL qualifiers are the safest spelling. +Format names are case-independent. `bgra8` is read as `rgba8`, which reads and writes the same values. Any other format, including valid GLSL ones the engine cannot allocate (`snorm`, `rg16f`, `rgba16`, 16-bit integers...), falls back to `rgba8` in the shader declaration and the allocation alike, with a warning shown in the shader's editor. ## Patterns diff --git a/docs/reference-manual/languages/glsl.md b/docs/reference-manual/languages/glsl.md index 84a1c025..913f7c98 100644 --- a/docs/reference-manual/languages/glsl.md +++ b/docs/reference-manual/languages/glsl.md @@ -67,10 +67,10 @@ Important Vulkan differences: Texture coordinates, framebuffer origins and clip coordinates are separate concerns. Do not fix a flipped result by adding an unconditional Y flip everywhere. -- **ISF fragments:** `isf_FragNormCoord` is the current normalized coordinate. `IMG_NORM_PIXEL` takes normalized coordinates; `IMG_PIXEL` takes pixel coordinates; `IMG_TEXEL` takes integer texel coordinates. `IMG_THIS_*` samples at the current fragment. These helpers account for runtime texture orientation. See the [full macro table]({{ site.baseurl }}/reference/languages/isf.html#texture-macros). +- **ISF fragments:** `isf_FragNormCoord` is the current normalized coordinate. `IMG_NORM_PIXEL` takes normalized coordinates; `IMG_PIXEL` takes pixel coordinates of the sampled image; `IMG_TEXEL` takes integer texel coordinates. `IMG_THIS_*` samples at the current fragment. These helpers account for runtime texture orientation; a raw `texture()` on an ISF image input reads the backend's own row order. See the [full macro table]({{ site.baseurl }}/reference/languages/isf.html#texture-macros). - **Compute images:** derive integer coordinates from `gl_GlobalInvocationID`, bounds-check against the resource size, then use `IMG_LOAD`, `IMG_STORE` or `IMG_STORE_LAYER`. These implement score's image-origin convention; raw `imageLoad`/`imageStore` do not apply its fixups. The [image contract]({{ site.baseurl }}/reference/languages/csf.html#images) covers dimensionality and access. - **Raw-raster vertices:** start `main()` with `isf_vertShaderInit()` and end with `isf_vertShaderFinish()`. Compute clip position with `clipSpaceCorrMatrix * VIEWPROJECTION_MATRIX * MODEL_MATRIX * position` for directly cabled geometry. The matrix and finish helper have complementary backend duties; do not call the helper twice or add another backend Y flip. Flattened scenes instead use [per-draw transforms]({{ site.baseurl }}/reference/languages/raw-raster.html#scene-path). -- **VSA:** write the original OpenGL-style clip-space position; score maps the depth convention. Do not add raw-raster init/finish calls or a second depth correction. +- **VSA:** write the original OpenGL-style clip-space position; score maps the depth convention. Do not add raw-raster init/finish calls or a second depth correction. Raw `texture()`, `textureLod()` and `texelFetch()` reads of a VSA image input have `v = 0` at the bottom on every backend, as in WebGL. For texture lookup outside fragment shading, choose an explicit LOD where required, as in the cookbook's compute audio recipe. Implicit fragment derivatives are not available to a compute invocation. Sampler filtering, wrapping and mip generation are separate choices described in the resource header. diff --git a/docs/reference-manual/languages/isf.md b/docs/reference-manual/languages/isf.md index 3ad8ca46..9e3c1961 100644 --- a/docs/reference-manual/languages/isf.md +++ b/docs/reference-manual/languages/isf.md @@ -17,13 +17,13 @@ This is the canonical shader contract for the [[ISF Shaders]] process. For GLSL The standard input vocabulary is `event`, `bool`, `long`, `float`, `point2D`, `color`, `image`, `audio`, and `audioFFT`. Standard multipass declarations include `TARGET`, `PERSISTENT`, `FLOAT`, `WIDTH`, and `HEIGHT`. Standard shader variables include `TIME`, `TIMEDELTA`, `DATE`, `FRAMEINDEX`, `PASSINDEX`, `RENDERSIZE`, and `isf_FragNormCoord`; the pixel/normalized image sampling helpers are part of ISF. -Do not assume another ISF host supports score's `point3D`, `audioHist`, cubemaps, volume inputs, depth companions, explicit sampler-state fields, `OUTPUTS`, `ALPHA`, `COMPOSITE`, or the additional pass fields `LAYER`, `Z`, `FORMAT`, and `PIPELINE_STATE`. Likewise, `PROGRESS`, `SAMPLERATE`, `isf_FragCoord`, `clipSpaceCorrMatrix`, `MSAA_SAMPLES`, `IMG_TEXEL`, `TEX_DIMENSIONS`, and `IMG_CUBE` are score runtime conveniences rather than a portable ISF baseline. `MODE` selects score shader families; [CSF]({{ site.baseurl }}/reference/languages/csf.html), [raw raster]({{ site.baseurl }}/reference/languages/raw-raster.html), and [VSA integration]({{ site.baseurl }}/reference/languages/vertex-shader-art.html) are not standard ISF rendering modes. +Do not assume another ISF host supports score's `point3D`, `audioHistogram`, cubemaps, volume inputs, depth companions, explicit sampler-state fields, `OUTPUTS`, `ALPHA`, `COMPOSITE`, or the additional pass fields `LAYER`, `FORMAT`, and `PIPELINE_STATE`. Likewise, `PROGRESS`, `SAMPLERATE`, `isf_FragCoord`, `clipSpaceCorrMatrix`, `MSAA_SAMPLES`, `IMG_TEXEL`, `TEX_DIMENSIONS`, and `IMG_CUBE` are score runtime conveniences rather than a portable ISF baseline. `MODE` selects score shader families; [CSF]({{ site.baseurl }}/reference/languages/csf.html), [raw raster]({{ site.baseurl }}/reference/languages/raw-raster.html), and [VSA integration]({{ site.baseurl }}/reference/languages/vertex-shader-art.html) are not standard ISF rendering modes. Header names become generated shader declarations. Use valid, distinct GLSL identifiers and do not redeclare generated uniforms, samplers or outputs in the body. Compatibility acceptance of `gl_FragColor` is score preprocessing, not Vulkan GLSL syntax. ## JSON header -Every shader starts with a JSON block enclosed in `/*{ ... }*/`. +The header is a JSON object in a block comment, `/*{ ... }*/`. ```glsl /*{ @@ -37,6 +37,28 @@ Every shader starts with a JSON block enclosed in `/*{ ... }*/`. }*/ ``` +### Header syntax + +These rules apply to every shader family: ISF, [CSF]({{ site.baseurl }}/reference/languages/csf.html), [raw raster]({{ site.baseurl }}/reference/languages/raw-raster.html), [Vertex Shader Art]({{ site.baseurl }}/reference/languages/vertex-shader-art.html) and geometry filters. + +- **Position.** The header is the first block comment whose content is a JSON object, so a licence or comment block may come before it. Of the comments that start with `{`, the first one that parses is the header; when none parses, the error of the first one is reported. +- **Comments and trailing commas.** `//` line comments and trailing commas before `}` or `]` are accepted in the header. Error lines and columns point into the source as written. +- **Case.** Keys are case-independent at every level (`"inputs"`, `"Name"`, `"pipeline_state"`...), as are `TYPE` values and keyword values such as `ACCESS`, `BUFFER_USAGE`, `VISIBILITY`, `FORMAT` and `MODE`. +- **Family.** The library and the file explorer read a file's family (its `MODE`) from the header only: a mode named in a comment or a string of the shader body does not change it. +- **Unknown types.** An `INPUTS` or `RESOURCES` entry whose `TYPE` the parser does not know, or whose `TYPE` is not a string, is an error naming the entry and listing the known types. +- **Messages.** An error about an entry is prefixed with the entry, e.g. `INPUTS[2] "gain": ...`; a blend error names the block it is in (`BLEND`, `BLEND_PER_ATTACHMENT[1]`...). Warnings are shown in the shader's editor and printed on the console. + +### Input names + +Each input name becomes a GLSL identifier in the generated code. A header is refused, with the name of the input, when an input name: + +- is a GLSL keyword, reserved word or type name (`layout`, `buffer`, `filter`, `input`, `output`, `half`, `vec4`, `sampler2D`...); +- starts with `gl_`, `isf_`, `ISF_` or `IMG_`, or contains two consecutive underscores; +- is a built-in name of the generated code: `TIME`, `TIMEDELTA`, `PROGRESS`, `SAMPLERATE`, `PASSINDEX`, `FRAMEINDEX`, `RENDERSIZE`, `DATE`, `MSAA_SAMPLES`, `NUM_VIEWS`, `VIEW_INDEX`, `LAYER_INDEX`, the camera matrices, `PRIMITIVE_ID`, `BARYCENTRIC`, `TEX_DIMENSIONS*`, `LAYER_TEXEL` or `main`; +- is used by another input, or by a control the engine adds to that shader family. + +Names are never renamed: rename the input in the header and in the code. + ### Common fields | Field | Description | @@ -47,10 +69,11 @@ Every shader starts with a JSON block enclosed in `/*{ ... }*/`. | `CATEGORIES` | Library categories. | | `INPUTS` | UI controls, texture inputs, audio textures, buffers, and uniforms. | | `PASSES` | Multipass render plan. | -| `OUTPUTS` | Explicit output textures: multiple color outputs, depth, layers, cubemaps, formats, MSAA. | +| `OUTPUTS` | Explicit output textures: multiple color outputs, depth, layers, formats, MSAA. | +| `PIPELINE_STATE` | Raster state of every pass; a pass's own `PIPELINE_STATE` overrides it field by field. See [Depth](#depth). | | `ALPHA` | Output alpha convention: `"straight"` or `"premultiplied"`. | | `COMPOSITE` | Default composition mode: `"over"`, `"add"`, `"multiply"`, `"screen"`, `"replace"`. | -| `EXTENSIONS` | GLSL extensions to require. | +| `EXTENSIONS` | Array of GLSL extension names to require, such as `"GL_KHR_shader_subgroup_basic"`. An entry that is not an extension name is an error; a duplicate, or an extension the generated stage already requires, is required once. | If `MODE` is omitted, the file is treated as an ISF shader. `MODE` is reserved for other shader families such as CSF and raw raster. @@ -61,15 +84,19 @@ Scalar inputs create UI controls and uniforms with the same name. | ISF type | GLSL type | Typical fields | |---|---|---| | `float` | `float` | `DEFAULT`, `MIN`, `MAX` | -| `bool` | `bool` | `DEFAULT` | +| `bool` | `bool` | `DEFAULT` (`true` / `false`, or a number: non-zero is `true`) | | `event` | `bool` for one frame | none | -| `long` | `int` | `VALUES`, `LABELS`, `DEFAULT` | +| `long` | `int` | `VALUES`, `LABELS`, `DEFAULT`, or `MIN`, `MAX`, `DEFAULT` | | `point2D` | `vec2` | `DEFAULT`, `MIN`, `MAX` | | `point3D` | `vec3` | `DEFAULT`, `MIN`, `MAX` | | `color` | `vec4` | `DEFAULT`, `MIN`, `MAX` | An `event` input fires for one rendered frame on `true` or an ossia impulse. Connect a trigger/message cable to it when an effect needs a reset or one-shot action; unlike a `bool`, it is not a latched switch. +A `long` with `VALUES` is a combo box over those values, named by `LABELS` when there are as many labels as values. A `long` without `VALUES` is an integer spin box over the bounds it gives (`MIN`, `MAX`, either or both; the integer range on a side it does not give), starting at `DEFAULT`. + +The rendered output starts from the values the controls show, including before the first values arrive from the execution engine, for example in previews. + Example: ```json @@ -94,6 +121,10 @@ Use the ISF sampling macros: vec4 c = IMG_THIS_NORM_PIXEL(inputImage); ``` +The `IMG_*` macros read an image the same way up on every backend, `v = 0` at the bottom. A raw `texture()` or `texelFetch()` on an image input reads the texture in the backend's own row order, which differs between Vulkan and the other backends: read image inputs through the [texture macros](#texture-macros). + +An `image` input that declares `ACCESS` or `FORMAT` is a storage image, read with `IMG_LOAD(name, pixel)` or `imageLoad(name, ivec2 pixel)`. Sampling it (`IMG_NORM_PIXEL`, `texture()`, `texelFetch()`...) is refused with a message naming the input; to sample the image instead, remove `ACCESS` and `FORMAT` from its entry. + ### 3D image / volume ```json @@ -112,7 +143,20 @@ Cubemaps are GLSL `samplerCube` values. Sample with `texture(environment, dir)` ### Sampleable depth -For inputs that carry a depth companion, set `DEPTH: true`. This exposes a depth sampler next to the color sampler where the upstream producer provides one. +An `image` or `cubemap` input marked `"IS_DEPTH": true` also binds the depth of what its producer renders, as a `_depth` sampler next to the color sampler: + +```json +{ "NAME": "scene", "TYPE": "image", "IS_DEPTH": true } +``` + +```glsl +float d = IMG_THIS_DEPTH(scene); // depth at this fragment +float d2 = IMG_DEPTH_NORM_PIXEL(scene, uv); // at a normalized coordinate +float d3 = IMG_DEPTH_PIXEL(scene, pixel); // at a pixel of the depth texture +float dc = IMG_CUBE_DEPTH(environment, dir); // depth of a cubemap input +``` + +`DEPTH` is not a depth flag: on a texture it is the depth of a volume. A volume (`DIMENSIONS: 3`) has no depth attachment, so `IS_DEPTH` on one is refused. The producer must declare that it writes depth, see [Depth](#depth). ### Sampler state @@ -124,15 +168,48 @@ Texture inputs can declare sampler state directly on the input object: "TYPE": "image", "FILTER": "nearest", "WRAP": "clamp_to_edge", - "MIPMAP_MODE": "linear", - "ANISOTROPY": 8.0 + "MIPMAP_MODE": "linear" } ``` -Supported fields include `WRAP`, `WRAP_S`, `WRAP_T`, `WRAP_R`, `FILTER`, `MIN_FILTER`, `MAG_FILTER`, `MIPMAP_MODE`, `BORDER_COLOR`, `COMPARE`, `ANISOTROPY`, `LOD_BIAS`, `MIN_LOD`, and `MAX_LOD`. +The sampler fields are `WRAP`, `WRAP_S`, `WRAP_T`, `WRAP_R`, `FILTER`, `MIN_FILTER`, `MAG_FILTER`, `MIPMAP_MODE` and `COMPARE`. `ANISOTROPY`, `BORDER_COLOR`, `LOD_BIAS`, `MIN_LOD`, `MAX_LOD` and the `mirror_once` wrap mode are accepted but have no effect, as the renderer has no such sampler setting; each one set produces a warning naming the input and the key. + +Each sampler field — filter, address mode, mipmaps — is resolved on its own: + +1. the value set on the texture inlet in the inspector, when the user set one; +2. otherwise the value the shader's JSON declares; +3. otherwise the default: linear filtering, clamp to edge, no mipmaps. + +`COMPARE` always comes from the shader: the inlet of a comparison sampler shows its Filter, Address mode and Mipmaps rows disabled. The first entry of each row leaves the field unset and is named after what it resolves to, for example **Shader: Nearest** or **Default: Linear**; see [[ISF Shaders]]. + +A mipmap mode other than none, from the inlet or from `MIPMAP_MODE`, also gives the input a mip chain, generated from what the upstream process renders. A `cubemap` input whose mipmap mode is left unset follows the texture bound to it: linear mipmapping when that texture has mip levels, none otherwise. Audio inputs only use `FILTER` and `WRAP`. +### Pointer inputs + +A shader can read the mouse and the touch history of a [[Window device]] through ordinary inputs. When the score has a window device, these inputs are bound to its addresses by default; the address shows on the inlet, where it can be changed or replaced by a cable: + +| Input | Bound to | +|---|---| +| `point2D` named `iMouse` | `cursor/absolute`: window pixels. | +| `point2D` whose name contains `mouse` (any case) | `cursor/gl`: `[0, 1]`, y up. In a [Vertex Shader Art]({{ site.baseurl }}/reference/languages/vertex-shader-art.html#mouse-and-touch) shader, `mouse` is bound to `cursor/clip` instead. | +| 2D `image` named `touch` | `touch/history`, read through a 32 × 240 RGBA32F target, nearest filtering, clamp to edge. | + +The touch history is the [vertexshaderart.com](https://www.vertexshaderart.com) `touch` texture, described in the [VSA reference]({{ site.baseurl }}/reference/languages/vertex-shader-art.html#mouse-and-touch). It stores time in alpha: an ISF reads it with `IMG_NORM_PIXEL_PREMULTIPLIED` / `IMG_PIXEL_PREMULTIPLIED`, since `IMG_NORM_PIXEL` divides by alpha (see [Texture macros](#texture-macros)). + +```json +"INPUTS": [ + { "NAME": "mouse", "TYPE": "point2D" }, + { "NAME": "touch", "TYPE": "image" } +] +``` + +```glsl +// Pointer 0 (mouse or first finger) this frame: x, y in clip space, pressure, time. +vec4 newest = IMG_NORM_PIXEL_PREMULTIPLIED(touch, vec2(0.5 / 32.0, 0.5 / 240.0)); +``` + ## Audio textures ISF audio inputs create GPU textures from audio data: @@ -140,9 +217,11 @@ ISF audio inputs create GPU textures from audio data: ```json { "NAME": "wave", "TYPE": "audio", "MAX": 512, "FILTER": "nearest" } { "NAME": "fft", "TYPE": "audioFFT", "MAX": 512 } -{ "NAME": "hist", "TYPE": "audioHist", "MAX": 512 } +{ "NAME": "hist", "TYPE": "audioHistogram", "MAX": 512 } ``` +`MAX` caps the texture width: a waveform is resampled linearly to `MAX` samples per channel, and an FFT or histogram row averages its bins into `MAX` columns. A row narrower than `MAX` keeps its own width. + Use `FILTER: "nearest"` when FFT bin accuracy matters. ## Texture macros @@ -151,20 +230,17 @@ Use the helpers instead of raw `gl_FragCoord` coordinate math. They keep orienta | Macro | Description | |---|---| -| `IMG_PIXEL(tex, coord)` | Sample at pixel coordinate. | +| `IMG_PIXEL(tex, coord)` | Sample at a pixel coordinate of `tex` itself: `coord / IMG_SIZE(tex)`. | | `IMG_NORM_PIXEL(tex, coord)` | Sample at normalized `[0, 1]` coordinate. | -| `IMG_THIS_PIXEL(tex)` | Sample at this fragment's pixel coordinate. | -| `IMG_THIS_NORM_PIXEL(tex)` | Sample at this fragment's normalized coordinate. | +| `IMG_THIS_PIXEL(tex)` | `IMG_PIXEL(tex, isf_FragCoord.xy)`: pixel-aligned, so an input of another size than the output is not stretched. | +| `IMG_THIS_NORM_PIXEL(tex)` | Sample at this fragment's normalized coordinate: the input is stretched over the output. | +| `IMG_NORM_PIXEL_LOD(tex, coord, lod)` | `IMG_NORM_PIXEL` at an explicit mip level. | | `IMG_TEXEL(tex, coord)` | Texel fetch at integer coordinate. | | `IMG_SIZE(tex)` / `TEX_DIMENSIONS(tex)` | Texture size in pixels. | | `IMG_CUBE(tex, dir)` | Cubemap sample. | +| `IMG_THIS_DEPTH`, `IMG_DEPTH_PIXEL`, `IMG_DEPTH_NORM_PIXEL`, `IMG_CUBE_DEPTH` | Depth of an `IS_DEPTH` input, see [Sampleable depth](#sampleable-depth). | -Textures coming from other nodes' render targets are premultiplied. There is no straight-alpha sampling macro: when a source must be unpremultiplied before color math, sample it normally and divide by alpha yourself: - -```glsl -vec4 c = IMG_NORM_PIXEL(inputImage, isf_FragNormCoord); -if (c.a > 0.0) c.rgb /= c.a; -``` +Textures coming from other nodes' render targets are premultiplied. On a 2D `image` input, `IMG_PIXEL`, `IMG_NORM_PIXEL`, `IMG_THIS_PIXEL`, `IMG_THIS_NORM_PIXEL` and `IMG_TEXEL` return straight color: they divide by alpha, and return `vec4(0)` where alpha is 0. Append `_PREMULTIPLIED` (`IMG_NORM_PIXEL_PREMULTIPLIED`, `IMG_PIXEL_PREMULTIPLIED`, `IMG_THIS_PIXEL_PREMULTIPLIED`, `IMG_THIS_NORM_PIXEL_PREMULTIPLIED`, `IMG_TEXEL_PREMULTIPLIED`) to read the stored values as they are, for example when alpha holds data rather than coverage. ## Built-in uniforms @@ -181,7 +257,9 @@ if (c.a > 0.0) c.rgb /= c.a; | `isf_FragNormCoord` | `vec2` | Current normalized fragment coordinate. | | `isf_FragCoord` | `vec4` | Platform-corrected fragment coordinate. | | `clipSpaceCorrMatrix` | `mat4` | Clip-space correction matrix for custom vertex shaders. | -| `MSAA_SAMPLES` | `int` | Active sample count when MSAA is enabled. | +| `MSAA_SAMPLES` | `int` | Sample count of the render target the pass draws into. | + +`DATE` and `SAMPLERATE` (the audio engine's sample rate) are set in every shader family, single-pass ISF included. ## Outputs @@ -205,7 +283,7 @@ void main() { ] ``` -Color outputs become `layout(location = N) out vec4 name;`. A depth output is written through `gl_FragDepth`. +Color outputs become `layout(location = N) out vec4 name;`. A depth output is written through `gl_FragDepth`, and only with the depth state of [Depth](#depth) declared. ### Output fields @@ -214,16 +292,13 @@ Color outputs become `layout(location = N) out vec4 name;`. A depth output is wr | `NAME` | Output name. | | `TYPE` | `"color"` or `"depth"`. Defaults to `"color"`. | | `FORMAT` | Texture format such as `rgba8`, `rgba16f`, `rgba32f`, `r32f`, `d32f`. | -| `WIDTH`, `HEIGHT` | Fixed size or expression. | +| `WIDTH`, `HEIGHT` | Fixed size or expression, see [Size expressions](#size-expressions). | | `LAYERS` | Texture array layer count. | -| `DEPTH` | 3D texture depth. | -| `CUBEMAP` | Allocate as cubemap. | -| `GENERATE_MIPS` | Generate mipmaps after rendering. | | `SAMPLES` | MSAA sample count. | | `ALPHA` | Per-output alpha convention. | | `COMPOSITE` | Per-output compositing mode. | -All color outputs in one pass share the render target size. Cubemap outputs are square. +All color outputs in one pass share the render target size. A classic ISF draws 2D (or layered) outputs: an output with `DEPTH` greater than 1, `CUBEMAP` or `GENERATE_MIPS` is refused. Use a [raw raster]({{ site.baseurl }}/reference/languages/raw-raster.html#fragment-outputs-and-render-targets) shader for cubemap and mipmapped outputs, and a [CSF]({{ site.baseurl }}/reference/languages/csf.html#3d-image) for volumes. ## Multipass rendering @@ -240,17 +315,48 @@ All color outputs in one pass share the render target size. Cubemap outputs are |---|---| | `TARGET` | Intermediate output to render. Missing target means final output. | | `PERSISTENT` | Preserve the target across frames. | -| `FLOAT` | Use floating-point storage. | -| `FILTER` | `"NEAREST"` for nearest sampling. | +| `FLOAT` | Use floating-point storage (`rgba32f`). | +| `FILTER` | `"nearest"` (any case) for nearest sampling. | | `WIDTH`, `HEIGHT` | Fixed size or expression. | | `LAYER` | Render into a texture-array layer. | -| `Z` | Render into a 3D texture slice. | -| `FORMAT` | Per-pass intermediate format. | -| `PIPELINE_STATE` | Per-pass raster state override. | +| `FORMAT` | Per-pass format: `rgba8`, `rgba16f`, `rgba32f`, `r8`, `r16f` or `r32f`; it takes precedence over `FLOAT`. Another value falls back to `rgba8` with a warning. | +| `PIPELINE_STATE` | Per-pass raster state, overriding the top-level `PIPELINE_STATE` field by field. | + +A pass targeting a 3D output or declaring `Z` is refused: write volumes with a [CSF]({{ site.baseurl }}/reference/languages/csf.html#3d-image). A classic ISF always draws its fullscreen quad: `VERTEX_COUNT` and `INSTANCE_COUNT` in a `PIPELINE_STATE`, top-level or per pass, are refused. Use a [raw raster]({{ site.baseurl }}/reference/languages/raw-raster.html#draw-counts) or [Vertex Shader Art]({{ site.baseurl }}/reference/languages/vertex-shader-art.html) shader to choose the draw counts. ISF v1 `PERSISTENT_BUFFERS` are converted to persistent passes internally. The legacy field accepts an array of target names or an object mapping names to `WIDTH`, `HEIGHT` and `FLOAT` settings. Each name must match a pass's `TARGET`; an unmatched name is ignored with a diagnostic. Prefer `PERSISTENT: true` on the pass in new shaders. -The last pass normally renders to the process output. Use `PASSINDEX` to branch between pass code paths. +Use `PASSINDEX` to branch between pass code paths. + +### The last pass + +The last pass draws straight into the destination, the render target of the node it is connected to, unless one of these gives it a target of its own: + +- it is `PERSISTENT`; +- its `WIDTH` / `HEIGHT` resolve to another size than the destination's; +- its `FORMAT` is single-channel (`r8`, `r16f`, `r32f`) and differs from the destination's. + +Such a target is then copied into the destination. `FLOAT`, a four-channel `FORMAT` and `FILTER` do not create a target: at the destination's size, one draw gives the same pixels either way. The same rules apply to a single-pass shader. A shader with `OUTPUTS` sizes its outputs itself. + +## Depth + +A shader that writes `gl_FragDepth`, or that declares a `depth` output, declares its depth state in the `PIPELINE_STATE` of its last pass (top-level, or that pass's own): + +```json +"PIPELINE_STATE": { "DEPTH_TEST": true, "DEPTH_WRITE": true, "DEPTH_COMPARE": "always" } +``` + +Without `DEPTH_TEST` and `DEPTH_WRITE`, the depth test of the fullscreen quad stays off and Vulkan and OpenGL write no depth; score prints a warning once for such a shader. A shader that writes no depth gets no depth target, attachment or extra pass. + +The declared depth reaches the node the shader is connected to, which reads it through an `IS_DEPTH` input (see [Sampleable depth](#sampleable-depth)): + +- a last pass drawn straight into the destination writes the destination's depth; +- a last pass with a target of its own (see [The last pass](#the-last-pass)) has its depth copied into the destination along with its color; +- each color output of a shader with `OUTPUTS` carries the shader's `depth` output into its destination. + +The copy depth-tests with the pass's `DEPTH_COMPARE`, as a pass drawn there would: several producers connected to one port depth-test against each other through that port's render target, in whichever order they are drawn. A `DEPTH_COMPARE` of `always` replaces the depth. When the destination has no depth attachment, the depth stays behind and a warning says so. The depth of the passes before the last one, and of a layered depth output, is not passed on. + +score's 3D rendering uses reverse-Z: the nearer fragment has the greater depth. `DEPTH_COMPARE` values are matched ignoring case, `_`, `-` and spaces (`less_equal`, `lessOrEqual`, `lequal`...); an unknown one is read as `greater`, with a warning. ## Size expressions @@ -260,7 +366,15 @@ The last pass normally renders to the process output. Use `PASSINDEX` to branch { "TARGET": "half", "WIDTH": "$WIDTH / 2", "HEIGHT": "$HEIGHT / 2" } ``` -Available variables include `$WIDTH`, `$HEIGHT`, `$inputName` for scalar controls, and `$WIDTH_inputImage` / `$HEIGHT_inputImage` for input texture sizes. Expressions support standard arithmetic and common functions such as `min`, `max`, `sqrt`, `ceil`, `floor`, `pow`, `sin`, and `cos`. +Available variables include `$WIDTH`, `$HEIGHT` (the render size), `$inputName` for scalar controls, and `$WIDTH_inputImage` / `$HEIGHT_inputImage` for input texture sizes. Pass sizes and `OUTPUTS` sizes are evaluated the same way; a target is rebuilt when a control or input size its expression reads changes. Expressions support standard arithmetic and common functions such as `min`, `max`, `sqrt`, `ceil`, `floor`, `pow`, `sin`, and `cos`. + +These are score's expression functions, the same as in the Math processes. Three of them take their arguments in another order than GLSL: + +| Expression | GLSL equivalent | +|---|---| +| `lerp(t, a, b)` | `mix(a, b, t)` | +| `step(value, edge)` | `step(edge, value)` | +| `smoothstep(value, start, stop)` | `smoothstep(start, stop, value)` | The texture inlet inspector also provides an explicit size checkbox with width and height controls. Uncheck it for **Auto** sizing; this is distinct from allocating a fixed-size target in the shader header. The **Show shader previews** toolbar action controls library and inspector previews. See [[Graphics pipeline]] for sizing and backend limits, and [[Shader cookbook]] for feedback, MRT and sampler recipes. diff --git a/docs/reference-manual/languages/raw-raster.md b/docs/reference-manual/languages/raw-raster.md index cac0a0d1..755df0cd 100644 --- a/docs/reference-manual/languages/raw-raster.md +++ b/docs/reference-manual/languages/raw-raster.md @@ -11,7 +11,7 @@ permalink: /reference/languages/raw-raster.html This is the canonical shader contract for the [[Render Pipeline]] process. For GLSL syntax, generated declarations and portability, see [GLSL runtime reference]({{ site.baseurl }}/reference/languages/glsl.html). For worked tasks, see [[Shader cookbook]]. -`RAW_RASTER_PIPELINE` is a score-specific format, not standard ISF. The fragment file contains the JSON header; the vertex file has the same stem (`.fs` / `.vs` or `.frag` / `.vert`). Declare resources and stage interfaces in the header rather than duplicating generated GLSL declarations. Shared controls and sampling helpers follow the [ISF runtime contract]({{ site.baseurl }}/reference/languages/isf.html); the raster interface below adds geometry, state and scene resources. +`RAW_RASTER_PIPELINE` is a score-specific format, not standard ISF. The fragment file contains the JSON header; the vertex file has the same stem (`.fs` / `.vs` or `.frag` / `.vert`). Declare resources and stage interfaces in the header rather than duplicating generated GLSL declarations. Shared controls and sampling helpers follow the [ISF runtime contract]({{ site.baseurl }}/reference/languages/isf.html), including its [header syntax]({{ site.baseurl }}/reference/languages/isf.html#header-syntax) and [input name rules]({{ site.baseurl }}/reference/languages/isf.html#input-names); the raster interface below adds geometry, state and scene resources. ## Minimal geometry shader @@ -73,7 +73,7 @@ void main() { | `MULTIVIEW` | Number of views for multiview rendering. | | `EXECUTION_MODEL` | Repeated raster invocation: per mip, per layer, per cubemap face, manual. | | `ALPHA`, `COMPOSITE`, `QUEUE`, `LAYER` | Alpha and compositing / transparent layering controls. | -| `EXTENSIONS` | GLSL extensions to require. | +| `EXTENSIONS` | GLSL extension names to require; see [ISF common fields]({{ site.baseurl }}/reference/languages/isf.html#common-fields). `GL_ARB_shader_draw_parameters`, and `GL_EXT_multiview` with `MULTIVIEW`, are always required. | | `CLIP_DISTANCES`, `CULL_DISTANCES`, `DEPTH_LAYOUT`, `PRIMITIVE_DATA` | Advanced raster features. | ## Vertex inputs @@ -144,8 +144,8 @@ Output fields: | `LAYERS` | Texture array layer count. | | `DEPTH` | 3D texture depth. | | `CUBEMAP` | Allocate as cubemap. | -| `GENERATE_MIPS` | Generate mipmaps after rendering. | -| `SAMPLES` | MSAA sample count. | +| `GENERATE_MIPS` | Allocate a mip chain and generate it after rendering. Color outputs only: refused on a depth output, which no backend can generate. | +| `SAMPLES` | MSAA sample count. The output is rendered multisampled and resolved into the texture, with a depth buffer at the same count. | | `ALPHA`, `COMPOSITE` | Per-output alpha / compositing policy. | 3D texture outputs cannot currently be written directly by raw raster; use [[Compute Shaders]] with `EXECUTION_MODEL: 3D_IMAGE` for true volumetric writes. @@ -180,6 +180,10 @@ vec4 t = IMG_TEXEL(albedo, ivec2(10, 20)); vec4 e = IMG_CUBE(env, reflectDir); ``` +Sampler fields resolve as in ISF, see [ISF sampler state]({{ site.baseurl }}/reference/languages/isf.html#sampler-state); an `image` with `ACCESS` or `FORMAT` is a storage image, whose sampled reads are refused as in ISF. A storage image the shader writes is allocated at its `WIDTH` / `HEIGHT`, literal or expression, and reallocated when they resolve to another size. Without a `VISIBILITY`, a storage image is visible to the fragment stage. + +A writable `storage` input whose `LAYOUT` ends in a flexible array gets a **` size`** control (default 1024), and is allocated with the std430 size of its fixed part plus that many elements. + ### Storage and uniform buffers ```json @@ -203,6 +207,10 @@ vec4 e = IMG_CUBE(env, reflectDir); `storage` uses SSBOs. `uniform` uses std140 UBOs. `VISIBILITY` controls which shader stages can access the binding. +A buffer cabled into one of these inputs, or matched by name from the upstream geometry, is bound at the range its producer publishes. A `uniform` block needs a buffer with uniform usage holding at least its fixed part plus one element of a trailing array; a `storage` block needs a buffer with storage usage; the offset must be a multiple of the device's buffer offset alignment. A buffer that does not qualify is not bound: the block reads zeros from its own placeholder, and a warning is printed once per change of reason. + +The process adds the **Mode** and blend controls (see [[Render Pipeline]]) to every raw raster shader: an input named `Mode`, `EnableBlend`, `SrcColor`, `DstColor`, `OpColor`, `SrcAlpha`, `DstAlpha` or `OpAlpha` is refused. + ## Built-in uniforms and macros | Name | Type | Description | @@ -216,7 +224,7 @@ vec4 e = IMG_CUBE(env, reflectDir); | `CAMERA_POSITION` | `vec3` | Active camera world position. | | `VIEW_INDEX` | `int` | Multiview view index. `0` when not multiview. | | `NUM_VIEWS` | `int` | Multiview count when multiview is enabled. | -| `MSAA_SAMPLES` | `int` | Active MSAA sample count. | +| `MSAA_SAMPLES` | `int` | Sample count of the render target this pass draws into: its output's `SAMPLES`, an iteration's own count, or the renderer's. | When a raw raster shader uses the built-in camera and does not declare its own `camera` binding, *score* injects a camera uniform block. The process gets a `Camera` geometry inlet when camera data can be connected. @@ -271,13 +279,26 @@ Top-level `AUXILIARY` declares buffers and textures that are resolved from upstr { "NAME": "scene_counts", "TYPE": "uniform", "LAYOUT": [ { "NAME": "drawCount", "TYPE": "uint" } ] }, { "NAME": "base_color", "TYPE": "texture", "FILTER": "linear" }, - { "NAME": "shadow_atlas", "TYPE": "image", "DEPTH": true, "COMPARE": "less" }, + { "NAME": "shadow_atlas", "TYPE": "image", "IS_DEPTH": true, "COMPARE": "less" }, { "NAME": "history", "TYPE": "storage_image", "FORMAT": "rgba16f", "ACCESS": "read_write", "WIDTH": 1024, "HEIGHT": 1024 } ] ``` -Auxiliary buffer entries default to SSBOs. Use `TYPE: "uniform"` for UBOs. Auxiliary texture types include `image`, `texture`, `cubemap`, `image_cube`, `storage_image`, `storage_cube`, `storage_image_array`, and `storage_3d`. +Auxiliary buffer entries default to SSBOs. Use `TYPE: "uniform"` for UBOs. Auxiliary texture types include `image`, `texture`, `cubemap`, `image_cube`, `storage_image`, `storage_cube`, `storage_image_array`, and `storage_3d`. A texture is a storage image only when its `TYPE` is one of the `storage_*` ones; a `FORMAT` alone does not make it one, and a `"STORAGE"` key is refused. `IS_DEPTH: true` marks a texture that holds depth (sampled with `COMPARE` for shadows); `DEPTH` is the depth of a volume. + +A writable storage image that nothing upstream provides belongs to the node: it is allocated at its `WIDTH` / `HEIGHT` (and `DEPTH` or `LAYERS`), resolved like the node's other size expressions, the render size filling a missing side. + +### Scene texture pool + +Behind a [[Scene Preprocessor]], the material textures travel as one shared pool, declared as auxiliary textures by name: + +| Name | Content | +|---|---| +| `materialArray0` … `materialArray7` | Texture arrays of the pool, one per texture size bucket. | +| `materialDyn0` … `materialDyn7` | Dynamic texture slots. | + +The per-channel names `baseColorArray`, `metalRoughArray`, `normalArray`, `emissiveArray`, `occlusionArray` and `baseColorDyn`, `metalRoughDyn`... are aliases of the same pool, kept for existing documents. They are deprecated: an input using one prints a warning once, naming `materialArray` / `materialDyn`. ## Pipeline state @@ -308,20 +329,35 @@ Common fields: | Field | Description | |---|---| -| `DEPTH_TEST`, `DEPTH_WRITE`, `DEPTH_COMPARE` | Depth state. | -| `DEPTH_BIAS`, `SLOPE_SCALED_DEPTH_BIAS` | Depth bias. | +| `DEPTH_TEST`, `DEPTH_WRITE`, `DEPTH_COMPARE` | Depth state. An unknown `DEPTH_COMPARE` is read as `greater` (reverse-Z), with a warning. | +| `DEPTH_BIAS`, `SLOPE_SCALED_DEPTH_BIAS` | Depth bias. `DEPTH_BIAS` is a whole number of depth units; use `SLOPE_SCALED_DEPTH_BIAS` for a fractional bias. | | `CULL_MODE` | `none`, `front`, or `back`. | | `FRONT_FACE` | `ccw` or `cw`. | | `POLYGON_MODE` | `fill` or `line`. | | `LINE_WIDTH` | Line width where supported. | -| `TOPOLOGY` | `triangles`, `triangle_strip`, `triangle_fan`, `lines`, `line_strip`, `points`. | -| `VERTEX_COUNT`, `INSTANCE_COUNT` | Procedural draw counts. | +| `TOPOLOGY` | `triangles` (or `triangle_list`), `triangle_strip`, `triangle_fan`, `lines` (or `line_list`), `line_strip`, `points`. | +| `VERTEX_COUNT`, `INSTANCE_COUNT` | Draw counts: non-negative integers or expressions, see [Draw counts](#draw-counts). | | `BLEND` | One blend state for all outputs, or boolean shortcut. | -| `BLEND_PER_ATTACHMENT` | Per-output blend states. | -| `STENCIL_TEST` and stencil masks / ops | Stencil state. | +| `BLEND_PER_ATTACHMENT` | Per-output blend states: exactly one entry per `FRAGMENT_OUTPUTS` entry (one for a shader without `FRAGMENT_OUTPUTS`). Another count is refused; use `BLEND` for a blend shared by every output. | +| `STENCIL_TEST` and stencil masks / ops | Stencil state. Masks are non-negative integers. | | `SHADING_RATE` | Variable-rate shading `[w, h]` where supported. | -If `VERTEX_COUNT` is set and `VERTEX_INPUTS` is empty, the pipeline performs a procedural draw driven by `gl_VertexIndex` / `gl_InstanceIndex`. If `VERTEX_INPUTS` is non-empty, the draw count is clamped to upstream geometry to avoid out-of-bounds reads. +Value names are matched ignoring case, `_`, `-` and spaces: `triangle_strip`, `TriangleStrip` and `triangle-strip` are the same. A negative or fractional count or mask is refused with the key's name. + +A blend entry takes `ENABLE`, `SRC_COLOR`, `DST_COLOR`, `OP_COLOR`, `SRC_ALPHA`, `DST_ALPHA`, `OP_ALPHA` and `COLOR_WRITE`; any other key, `SRC` / `DST` / `OP` included, is refused. + +The 3D render targets of score have no stencil aspect: a pipeline with `STENCIL_TEST` drawn into a target without a stencil attachment runs without the stencil test, with a warning naming the shader. + +### Draw counts + +`VERTEX_COUNT` makes a procedural draw: `VERTEX_COUNT` vertices and `INSTANCE_COUNT` instances driven by `gl_VertexIndex` / `gl_InstanceIndex`, reading no mesh. It requires `"VERTEX_INPUTS": []`; next to vertex inputs it is refused. + +`INSTANCE_COUNT` alone, on a shader that draws the mesh, multiplies the instance count of each of the mesh's draws. It is not applied to GPU-indirect draws, whose counts the host never reads, nor to a mesh with per-instance attributes, whose data only covers its own instances; a warning says so. + +### Backend limits + +- Direct3D and Metal have no triangle fans. A fan whose vertex order the CPU knows (non-indexed, or with a CPU index buffer) is drawn as the equivalent triangle list, on every backend. A fan it cannot rewrite — a procedural `TOPOLOGY: "triangle_fan"`, a GPU index buffer, an indirect draw — is not drawn on those backends, with one warning per pipeline. +- Metal has no cull distances: a shader declaring `CULL_DISTANCES` is not drawn on Metal, with one warning, and its consumers receive an empty texture. ## Alpha, compositing, and transparency @@ -380,7 +416,9 @@ Raw raster normally draws once per frame. `EXECUTION_MODEL` repeats the raster p | `PER_LAYER` | Draw once per texture-array layer. | | `MANUAL` | Draw `COUNT` times; `COUNT` is an expression. | -`MULTIVIEW` can render several layers / views in a single draw when the backend supports it. It is commonly used with `CUBEMAP: true` and `MULTIVIEW: 6` for camera-array or IBL passes. +Each `PER_MIP`, `PER_LAYER` or `PER_CUBE_FACE` iteration draws into every output the shader declares: the iterated level, layer or face of `TARGET`; the matching level or layer of another output with the same shape; the whole output otherwise; or, when none fits, a scratch texture, with a warning. Iterations honour each output's `SAMPLES`: they render multisampled and resolve into their level, layer or face. A depth output is written by the iterations whose size, shape and sample count match it — level 0 of a `PER_MIP` pass for a 2D depth, layer *i* of a `PER_LAYER` pass for a depth array with as many layers (Qt 6.12 or newer) — and the other iterations use a scratch depth buffer, with a warning. + +`MULTIVIEW` can render several layers / views in a single draw when the backend supports it. It is commonly used with `CUBEMAP: true` and `MULTIVIEW: 6` for camera-array or IBL passes. `MULTIVIEW: N` needs at least `N` layers on every color output (a cubemap has 6); fewer is refused. Where the backend lowers multiview to one draw per view, the draw runs once per view. Multiview iterations render single-sampled. ## Primitive data @@ -411,9 +449,9 @@ Advanced depth / culling fields: } ``` -- `CLIP_DISTANCES`: declares `gl_ClipDistance[N]` for user clip planes. -- `CULL_DISTANCES`: declares `gl_CullDistance[N]` for per-primitive culling. -- `DEPTH_LAYOUT`: conservative-depth qualifier for `gl_FragDepth`: `any`, `greater`, `less`, or `unchanged`. +- `CLIP_DISTANCES`: declares `gl_ClipDistance[N]` for user clip planes, `N` from 1 to 8. +- `CULL_DISTANCES`: declares `gl_CullDistance[N]` for per-primitive culling, `N` from 1 to 8. Not available on Metal, see [Backend limits](#backend-limits). +- `DEPTH_LAYOUT`: conservative-depth qualifier for `gl_FragDepth`: `any`, `greater`, `less`, or `unchanged`, in any case. Another value is refused. ## Examples diff --git a/docs/reference-manual/languages/vertex-shader-art.md b/docs/reference-manual/languages/vertex-shader-art.md index 17e318fd..170554bb 100644 --- a/docs/reference-manual/languages/vertex-shader-art.md +++ b/docs/reference-manual/languages/vertex-shader-art.md @@ -25,13 +25,17 @@ Common VSA fields: | Field | Description | |---|---| -| `POINT_COUNT` | Number of vertices to generate. | -| `PRIMITIVE_MODE` | `POINTS`, `LINES`, `LINE_STRIP`, or compatible VSA primitive mode. | +| `POINT_COUNT` | Default of the **Vertex Count** control (3 to 100000). Without a positive `POINT_COUNT` it is 3: one triangle. | +| `PRIMITIVE_MODE` | Default of the **Primitive Type** control: `POINTS`, `LINE_STRIP`, `LINE_LOOP`, `LINES`, `TRI_STRIP`, `TRI_FAN` or `TRIANGLES`, in any case. Without one, or with an unknown one (which prints a message), `TRIANGLES`. | | `LINE_SIZE` | Line / point sizing policy used by imported VSA shaders. | | `BACKGROUND_COLOR` | Clear color. | -| `INPUTS` | ISF-style controls, textures, and audio inputs. | +| `INPUTS` | ISF-style controls, images, and audio inputs. | -The process also supports the standard ISF metadata fields such as `DESCRIPTION`, `CREDIT`, `CATEGORIES`, and `ISFVSN`. +The process also supports the standard ISF metadata fields such as `DESCRIPTION`, `CREDIT`, `CATEGORIES`, and `ISFVSN`; without a `DESCRIPTION`, the description is "Vertex Shader Art". The header follows the [ISF header syntax]({{ site.baseurl }}/reference/languages/isf.html#header-syntax) and [input name rules]({{ site.baseurl }}/reference/languages/isf.html#input-names); `vertexCount` and `primitiveType` are the names of the two controls above. + +`LINE_LOOP` draws a closed polyline through the vertices, the last one joined to the first. `TRI_FAN` draws a fan; on backends without triangle fans (Direct3D, Metal), the same triangles are drawn as a list. + +VSA has no storage buffers, uniform blocks, geometry or storage images: such entries in `INPUTS` are ignored, with a message, and create no port. `event` inputs fire for one frame, as in ISF. `DATE`, `SAMPLERATE` and the other ISF built-in uniforms are available. ## Shader variables @@ -53,6 +57,35 @@ resolution // render size ISF-style controls declared in `INPUTS` are available as uniforms. +Image inputs can be read with the WebGL functions VSA art uses, `texture()`, `textureLod()` and `texelFetch()`, as well as with the ISF `IMG_*` macros. On a 2D image input, these raw reads are oriented like `IMG_NORM_PIXEL`: `v = 0` is the bottom of the image on every backend. A `v` on the edge between two rows, such as `n / 240.0`, reads the same row on every backend: the row above the edge, `floor(v * height)`. Other sampling functions (`textureGrad`, `textureOffset`, `textureProj`...) are left as written. + +### Mouse and touch + +Two inputs reproduce the `mouse` and `touch` of [vertexshaderart.com](https://www.vertexshaderart.com): + +```json +"INPUTS": [ + { "NAME": "mouse", "TYPE": "point2D" }, + { "NAME": "touch", "TYPE": "image" } +] +``` + +| Input | Default binding | Content | +|---|---|---| +| `mouse` | `cursor/clip` of the first [[Window device]] | Mouse position in clip space, x and y in `[-1, 1]`, y up. | +| `touch` | `touch/history` of the first window device | The last 240 frames of the window's pointers, a 32 × 240 RGBA float texture read with nearest filtering and clamp to edge. | + +In the touch history, column *c* is pointer *c*: column 0 is the mouse or the first finger, the next columns the further fingers in the order they came down. Row 0, at `v = 0`, is the newest frame; `v = 1` is 239 frames ago. Each texel is `(x, y, pressure, time)`: the position in clip space, y up, the pressure (1 while the mouse button is pressed; a released pointer keeps its position with pressure 0), and the time of the row, in seconds since the history started. + +```glsl +// Pointer 0, five frames ago. +vec4 t = texture(touch, vec2(0.5 / 32.0, (5.0 + 0.5) / 240.0)); +vec2 position = t.xy; +float pressure = t.z; +``` + +These bindings are set on the inlets, where they show and can be changed: rebind them to another address, or cable a texture into `touch`. They are made the same way in ISF and render pipeline shaders, see [ISF pointer inputs]({{ site.baseurl }}/reference/languages/isf.html#pointer-inputs). Without a window device, nothing is bound: an unconnected `touch` reads an empty texture. + VSA clip-space depth follows its original OpenGL-style convention; the runtime maps it to the backend's clip range. Do not add the raw-raster `isf_vertShaderInit()` / `isf_vertShaderFinish()` sequence or a second backend depth correction to a VSA shader. To understand how individual vertices make an image, start with the triangle recipe in [[Shader cookbook]]. It places three vertices and gives them one color; changing a vertex position changes the silhouette without loading a mesh. Point and line sizes remain GPU/backend-dependent. diff --git a/docs/reference-manual/processes/library/asset-loader.md b/docs/reference-manual/processes/library/asset-loader.md index 38b61315..3ac16025 100644 --- a/docs/reference-manual/processes/library/asset-loader.md +++ b/docs/reference-manual/processes/library/asset-loader.md @@ -57,7 +57,7 @@ The scene path preserves information that plain geometry cannot express: hierarc A typical custom renderer uses these stages: 1. **Asset Loader** reads the file and outputs a scene. -2. **Scene Filter** keeps visible nodes, while **Scene Graph Filter** and related [[Scene tools]] select by node paths, names, components and material properties. Use separate transform and material processes to modify those properties. +2. **Scene Filter** keeps visible nodes or nodes whose name matches a pattern, while **Scene Graph Filter** and related [[Scene tools]] select by node paths, names, components and material properties. Use separate transform and material processes to modify those properties. 3. **Scene Preprocessor** flattens the scene into GPU-resident geometry and auxiliary data. 4. **Render Pipeline** draws the flattened scene with a raw raster shader. diff --git a/docs/reference-manual/processes/library/compute-shader.md b/docs/reference-manual/processes/library/compute-shader.md index 14b5829c..7207e063 100644 --- a/docs/reference-manual/processes/library/compute-shader.md +++ b/docs/reference-manual/processes/library/compute-shader.md @@ -20,7 +20,7 @@ A CSF file is a GLSL compute shader with a JSON header. It can be dropped from t ## Using the process -Drop a CSF file into the score, connect its generated resource inlets, and send image outputs to a video output. Geometry outputs need [[Model Display]] or [[Render Pipeline]] before they can be displayed as an image. For geometry with its own topology, set the downstream Render Pipeline **Mode** to **Geometry**. +Drop a CSF file into the score, connect its generated resource inlets, and send image outputs to a video output. A CSF declares its controls and resources in `RESOURCES`, where `image` is a storage image and `texture` a sampled one; a shader still using `INPUTS` works, with a deprecation warning. A writable buffer ending in a flexible array gets a **size** control, and a `long` control without `VALUES` is a spin box over its `MIN` / `MAX`. A shader that cannot be used leaves the process without ports and logs a warning naming the process in the **Messages** panel. Geometry outputs need [[Model Display]] or [[Render Pipeline]] before they can be displayed as an image. For geometry with its own topology, set the downstream Render Pipeline **Mode** to **Geometry**. The [CSF language reference]({{ site.baseurl }}/reference/languages/csf.html) contains a [minimal image filter]({{ site.baseurl }}/reference/languages/csf.html#minimal-file), every resource and pass field, buffer layouts, geometry macros, expressions and built-ins. [[Shader cookbook]] provides image, geometry, indirect-work and audio recipes. @@ -59,7 +59,7 @@ Go directly to a relocated section: ## Runtime requirements -Compute and storage images require a compatible device and backend. Integer storage formats require Qt 6.10 or newer as well as device support; earlier Qt versions can fall back to RGBA8 without preserving integer semantics. See [[Graphics pipeline]]. Keep the bounds checks in the reference examples when changing image dimensions or geometry counts. +Compute and storage images require a compatible device and backend. Integer storage formats require Qt 6.10 or newer as well as device support; earlier Qt versions can fall back to RGBA8 without preserving integer semantics. A storage buffer can be as large as the device binds; a buffer that cannot be allocated, or a pass needing more workgroups than the device dispatches, is skipped with a warning rather than run partially. See [[Graphics pipeline]]. Keep the bounds checks in the reference examples when changing image dimensions or geometry counts. ## Related Processes diff --git a/docs/reference-manual/processes/library/model-display.md b/docs/reference-manual/processes/library/model-display.md index a62a9ca1..910721b8 100644 --- a/docs/reference-manual/processes/library/model-display.md +++ b/docs/reference-manual/processes/library/model-display.md @@ -32,7 +32,7 @@ Use [[Render Pipeline]] instead when you need custom vertex / fragment shaders, | Position | Camera position. Default looks from `(-1, -1, -1)` toward the center. | | Center | Camera look-at target. | | FOV | Field of view in degrees. | -| Near / Far | Near and far clipping planes. | +| Near / Far | Near and far clipping planes. Near: 0.001 to 1000, default 0.001. Far: 0.001 to 100000, default 1000. | | Camera | Projection model: Perspective, Fulldome equidistant, Fulldome equisolid, Fulldome stereographic, or Fulldome orthographic. | Fulldome modes render fisheye-style output for dome projection workflows. @@ -43,7 +43,7 @@ Fulldome modes render fisheye-style output for dome projection workflows. |---|---| | Tex. Proj. | Texture projection mode. | | Mode | Triangles, Points, or Lines. | -| Enable blend and blend factors | Optional fixed-function alpha blending. | +| Enable blend, Src Color, Dst Color, Op Color, Src Alpha, Dst Alpha, Op Alpha | Optional fixed-function blending. The operations are Add, Subtract, Reverse Subtract, Min and Max. | Texture projection modes include: diff --git a/docs/reference-manual/processes/library/render-pipeline.md b/docs/reference-manual/processes/library/render-pipeline.md index 17835407..5089e4aa 100644 --- a/docs/reference-manual/processes/library/render-pipeline.md +++ b/docs/reference-manual/processes/library/render-pipeline.md @@ -18,6 +18,19 @@ Use it when you need explicit control over vertex attributes, uniforms, buffers, For a cabled geometry's own topology, select **Geometry** in the **Mode** control. **Triangles**, **Points**, and **Lines** deliberately override it. A shader's explicit `PIPELINE_STATE.TOPOLOGY` takes precedence over the control. This matters particularly for CSF-generated point clouds and line strips. +Every Render Pipeline has these controls besides the ones its shader declares: + +| Control | Description | +|---|---| +| Mode | Triangles, Points, Lines or Geometry, as above. | +| Enable blend | Turns on the blend below. Off by default. | +| Src Color, Dst Color | Color blend factors (Zero, One, SrcColor, OneMinusSrcColor, DstColor, OneMinusDstColor, SrcAlpha, OneMinusSrcAlpha, DstAlpha, OneMinusDstAlpha, the constant factors and SrcAlphaSaturate). Default SrcAlpha and OneMinusSrcAlpha. | +| Op Color | Add, Subtract, Reverse Subtract, Min or Max. Default Add. | +| Src Alpha, Dst Alpha | Alpha blend factors. Default One and OneMinusSrcAlpha. | +| Op Alpha | As Op Color, for alpha. Default Add. | + +Image inputs named `touch` and `point2D` inputs named like `mouse` are bound to the window device's pointer addresses by default, as in ISF; see [ISF pointer inputs]({{ site.baseurl }}/reference/languages/isf.html#pointer-inputs). + ## Files A raw raster pipeline uses two files with the same base name: diff --git a/docs/reference-manual/processes/library/scene-tools.md b/docs/reference-manual/processes/library/scene-tools.md index 27fe02a9..d6f4d826 100644 --- a/docs/reference-manual/processes/library/scene-tools.md +++ b/docs/reference-manual/processes/library/scene-tools.md @@ -31,12 +31,14 @@ Only the controls relevant to the selected mode are used. Start with Inspector o ## Scene Filter -This smaller filter exposes only **pass through** and **keep visible only**. It is not a script editor and does not have the path/material selection controls of Scene Graph Filter. Its ports are **Scene In** and **Scene Out**. +This smaller filter has three modes: **pass through**, **keep visible only** and **keep matching names**. The last keeps the subtrees whose node name matches the **Name** pattern, written like an OSC address part — `*` and `?` wildcards, `[0-9]` ranges and `{wheel,door}` alternatives — and matched against the whole name. A matching node is kept with everything under it; its parents stay, so that it keeps its place and transform, but lose their own meshes and other components. An empty pattern keeps everything. It does not have the path/material selection controls of Scene Graph Filter. Its ports are **Scene In** and **Scene Out**. ## Flattened Scene Filter Place this process **after Scene Preprocessor**. It accepts **Geometry In** and emits **Geometry Out**, selecting flattened draws by equality or inequality of tag, material index, blend, depth-write, cull mode or topology. Set the integer **Match** for those modes. The format-ID modes instead compare the **Format ID** string. Use two branches to separate render passes without re-importing an asset. +The Scene Preprocessor merges the regular draws of a scene into one geometry. The tag, material index and format-ID modes select individual draws inside it: each draw is judged on its own `per_draws` record (its material tag and its `material_index` slot), and the kept draws are drawn from a command list. An instanced draw carries the tag of its prototype's material. The other modes keep or drop whole geometries. Filters can be chained. + ## Scene Selector Select a subtree **By path**, **By name** or **By index**, using **Path / Name** or **Index**. Index addresses the root list, starting at zero. **Rebase** is either Preserve transform or Zero out. Preserve retains the selected node's local transform, not the transforms of removed ancestors; Zero out removes its own transform for later placement with Transform 3D or Scene Group. diff --git a/docs/reference-manual/processes/library/shader.md b/docs/reference-manual/processes/library/shader.md index 7c49c1a5..ad6b1933 100644 --- a/docs/reference-manual/processes/library/shader.md +++ b/docs/reference-manual/processes/library/shader.md @@ -31,11 +31,13 @@ Use ISF when the shader renders a fullscreen image pass. Use a different process ## Files and live editing -An ISF shader is usually a `.fs` or `.frag` file with a JSON header followed by GLSL. A custom vertex shader can be paired with the fragment shader; the editor can live-code both stages. +An ISF shader is a `.fs` or `.frag` file with a JSON header followed by GLSL. A custom vertex shader can be paired with the fragment shader: a `.vs` / `.vert` file with the same name next to it is used as its vertex stage. The editor can live-code both stages. A `.glsl` file dropped in the score is not an ISF shader: it is handled as a [geometry filter]({{ site.baseurl }}/reference/languages/geometry-filter.html), and an include file that is not one makes no process. Press {% include shortcut.html content="Ctrl+Enter" %} in the shader editor to recompile during playback. For production authoring, the official ISF editor remains useful, but *score* supports the runtime-specific features documented here. -Shader sources can use `#include`; include paths are resolved before parsing. +Shader sources can use `#include`; include paths are resolved before parsing. Compiling a shader again picks up the changes of the files it includes. + +When a shader process loads a shader that cannot be used — a header or GLSL error — it has no ports, and a warning naming the process and the error is shown in the **Messages** panel. The editor shows the same error while typing. ## Language reference @@ -69,7 +71,21 @@ Go directly to a relocated section: ## Connecting and previewing -Connect a video or image producer to the generated image inlet, adjust the controls, and connect the shader output to a video output or another effect. Generators need no image inlet. The texture inlet inspector provides an explicit size checkbox with width and height controls; uncheck it for **Auto** sizing. The **Show shader previews** toolbar action controls library and inspector previews. See [[Graphics pipeline]] for sizing and backend limits. +Connect a video or image producer to the generated image inlet, adjust the controls, and connect the shader output to a video output or another effect. Generators need no image inlet. The **Show shader previews** toolbar action controls library and inspector previews. See [[Graphics pipeline]] for sizing and backend limits. + +The texture inlet inspector has these rows: + +| Row | Description | +|---|---| +| Size | Explicit render size of the input, with width and height controls; uncheck it for **Auto** sizing. | +| Format | Render target format: **Auto** (RGBA8, or the texture the upstream process publishes when the inlet can read it directly), RGBA8, RGBA16F, RGBA32F, R8, R16, R16F or R32F. | +| Filter | Nearest or Linear. | +| Address mode | Repeat, Clamp to edge or Mirror. | +| Mipmaps | None, Nearest or Linear. A mipmap mode gives the input a mip chain. | + +The first entry of Filter, Address mode and Mipmaps leaves the field to the shader: it reads **Shader: …** when the shader's JSON declares that field, **Default: …** otherwise, with the value it resolves to. A value picked in the inspector takes precedence over the shader's. When the shader samples the input with a depth comparison (`COMPARE`), these three rows are disabled. See [ISF sampler state]({{ site.baseurl }}/reference/languages/isf.html#sampler-state). + +An input whose name the shader reserves for pointers (`mouse`, `iMouse`, `touch`) is bound to the window device's mouse and touch addresses by default; see [ISF pointer inputs]({{ site.baseurl }}/reference/languages/isf.html#pointer-inputs). ## Related Processes diff --git a/docs/reference-manual/processes/library/vsa-shader.md b/docs/reference-manual/processes/library/vsa-shader.md index aff32516..6f739aa9 100644 --- a/docs/reference-manual/processes/library/vsa-shader.md +++ b/docs/reference-manual/processes/library/vsa-shader.md @@ -21,7 +21,9 @@ Use VSA for lightweight point clouds, line art, particle-like procedural visuals ## Authoring a visual -Use the [Vertex Shader Art reference]({{ site.baseurl }}/reference/languages/vertex-shader-art.html) for header fields, vertex outputs, time and resolution variables, and a complete [rotating spiral]({{ site.baseurl }}/reference/languages/vertex-shader-art.html#example). Begin with the three-vertex recipe in [[Shader cookbook]] to see how positions and colors form the image. Connect the resulting texture to a video output. Point and line sizes depend on the GPU/backend. +Use the [Vertex Shader Art reference]({{ site.baseurl }}/reference/languages/vertex-shader-art.html) for header fields, vertex outputs, time and resolution variables, and a complete [rotating spiral]({{ site.baseurl }}/reference/languages/vertex-shader-art.html#example). Begin with the three-vertex recipe in [[Shader cookbook]] to see how positions and colors form the image. Connect the resulting texture to a video output. Point and line sizes depend on the GPU/backend. The process adds a **Vertex Count** and a **Primitive Type** control, whose defaults come from the shader's `POINT_COUNT` and `PRIMITIVE_MODE`. + +A shader's `mouse` and `touch` inputs follow vertexshaderart.com: they are bound by default to the window device's `cursor/clip` and `touch/history` addresses, shown on the inlets, where they can be changed or cabled. See [mouse and touch]({{ site.baseurl }}/reference/languages/vertex-shader-art.html#mouse-and-touch). Go directly to a relocated section: