Shader usage - Windows platform

[kornellapu]

Hey everyone!

First of all thank you for this project!

I just started to use Flame Engine some weeks ago.
I am coming from a programmer/engineer perspective.

I would like to know is it possible to use SpriteComponent and custom shaders together?
I am trying to create an outline around sprite images (transparent background) with a custom shader.

First I could not built shaders I found, the build process just stucked with impellerc.exe without any trace (found out with verbose build).
The issue was there some error in the shader code I assume.

I went to test out with simpler shaders like a simple solid color shader. (The build did not stuck, that is why I assume I did some errors previously.)
I am setting the paint.shader property of SpriteComponent but seems like it is not working or does not have an effect on the image.

Is there something which I am not aware of with custom shaders and paint or with the whole pipeline?

Can you provide a full working example on windows how I should set up the project and create a compatible shader and use it on a single SpriteComponent image (which has a transparent background and and solid pixels as a drawing).
Or point out to documentation or to an existing tutorial?

I can provide you more details if necessary ^.^

Thank you for any information in advance!
Have a great day!
Kornél

[spydon]

Hi,
You don't want to use a shader on the paint object, what you want to do is postprocessing with a shader. Wrap your component in a PostProcessComponent and add your shader in there:
https://docs.flame-engine.org/main/flame/post_processing.html#postprocesscomponent

[kornellapu]

Thank you for your quick answer!
I will definitely try out!

[kornellapu]

Thank you for pointing out towards the solution!
I managed to create custom outline shader and use it in the code. Working fine ^.^

tldr;

[mind the postion, size property of PostProcessingComponent]
[mind the position where the underlying canvas is rendered on the screen in PostProcess inside postProcess function]

---

Details:

I followed this example here:
https://docs.flame-engine.org/latest/flame/post_processing.html#creating-a-custom-post-process

I had to modify the example code to have PostProcessingComponent deeper in the render hierarchy.
I needed the PostProcessingComponent exactly before (parenting) the shader affected SpriteComponent.

The detached class from world object came out like this (this is the wrapper class):

class ActionIconPostProcess extends PostProcessComponent{
  ActionIconPostProcess({required cardAction, super.position, super.anchor}) :
  super(
    postProcess: OutlinePostProcess(),  
    children: [ActionIconSprite(cardAction: cardAction)]
  );
}

I also created the ActionIconSprite class extending the SpriteComponent.
This class is handling the image sprites based on the cardAction parameter which comes from the game model.

Created the OutlinePostProcess class extending the PostProcess.
This class handling the default settings of the shader, creating the shader program, and setting the uniforms.

Also I encountered slight issues with canvas positioning/size and the anchor properties.
I think I am missing something here, but some slight changes in the overridden void postProcess(Vector2 size, Canvas canvas) function created the expected effect.

First of all I was not aware the PostProcessingComponent have a fixed position, scale and anchor property.
Luckily I had a draggable SpriteComponent and I could find the invisible "window" where the effect were applied.
Not overlapping with the PostProcessingComponent boundaries cause the SpriteComponent to disappear.
(The sampled texture was empty there.)

Creating a full-screen PostProcessingComponent (with ```HasGameRef```` listener for handling resize) solved the issue, but I felt like I was wasting resources.
Restructuring the render hierarchy solved the undraggable issue (draggable component is parent of the post processing component which wraps the shaded sprite component).

I was using Anchor.center generally everywhere in the SpriteComponents.
Also I calculated previously the middle point where the post processed sprite component should be.
After rendering the shaded image only one quarter (bottom right) of the sprite was visible.
I modified the shader to let me debug its limits and then noticed, the sprite was offseted to the canvas which were rendering the sprite.
I tried to set the anchor properties of components, but for me it seems it does not had an effect in neither combination.

The original part of the code from the tutorial (https://docs.flame-engine.org/latest/flame/post_processing.html#example-pixelation):

...
    canvas
      ..save()
      ..drawRect(Offset.zero & size.toSize(), myPaint)
      ..restore();
...

I modified this part of the code to translate the canvas to simulate centering:

...
    canvas
      ..save()
      ..translate(-size.x/2, -size.y/2) //centering
      ..drawRect(Offset.zero & size.toSize(), myPaint)
      ..restore();
...

After that everything worked fine as expected.
Setting only the parameters of drawRect to something like:
..drawRect(Rect.fromCenter(center: Offset.zero, width: size.x, height: size.y), myPaint)
did not worked, because it pulled some part of the sampled texture.

I ended up some hierarchy like this: (parent > children)
Word > draggable SpriteComponent > PostProcessingComponent > shaded SpriteComponent.

---

Thank you again for the point out!
Also I hope this explanation will be helpful to someone else too.

Have a great day!
Kornél

[spydon]

A clarification about the anchor;
The anchor is not related to how the canvas is positioned within the components render method, it only says that it's where in the component that the position should be when it is placed in its parent (and also what point the component rotates around). Inside of render you always have 0,0 in the top left corner of the bounding box of the component.
You can read more about how the anchor works here.