Colored Outline Shader in Unity

It seems like I’m on a roll with outline shaders! I’ve already written a basic cel shader with outline tutorial and an animated dotted outline shader, and now I’m doing this colored outline.

The special thing about the color of this outline is that it’s using the color of the texture, so that the outline color always matches the pixel it’s closest to. I find it’s a common tool used by 2D artists to make outlines feel more natural, as opposed to using a solid color like black for the whole outline.

Here’s an example of the difference it makes with an example from 2D pixel art from u/croakiee on Reddit:

This tutorial teaches you how to do this effect in 3D.

IMPORTANT: Since I already went over the basics of how to do the outline here, I’m not going to repeat that explanation in this tutorial. Do the basic tutorial first if you’re not sure how to create an outline at all.

Here’s the full code for the shader for reference, under a non-commercial open-source license:

->> Link to full code for Unity Colored Outline Shader

Also, The 3D model is from this free asset pack.

Now, on with the tutorial!

Colored Outline

This effect is quite simple, and happens completely inside the vertex shader.

If you remember from the basic outline tutorial, our outline is created by scaling the original model along each vertex’s normal direction. Now, to make the outline color the same as the vertex color, we simply sample the vertex color and add it to the vertex shader output.

Firstly, add a color to your vertex output struct:

struct vertexOutput {
    float4 pos : SV_POSITION;
    float4 color : TEXCOORD0;
}

And add one extra line of code in the vertex shader to sample the color:

output.color = tex2Dlod(_MainTex, float4(input.texCoord.xy, 0, 0));

And your output should look like this:

Why sample the color at each vertex, you might ask, instead of at each fragment like we usually do? Basically, this is a lazy way of blurring the outline color. The vertex shader is run less often (only for every vertex on the mesh) than the fragment shader (basically for every pixel), and the colors between each fragment are interpolated, which essentially blurs away details. If we sample at every fragment instead, we get too much detail, and the outline ends up looking like this:

Now that you’ve got your outline sampling the vertex color, let’s make the outline distinguishable from the model’s texture. As you can see right now, they’re a little bit too similar.

To do this, we’re going to add an outline color property and multiply our sampled color by the outline color. 

output.color = tex2Dlod(_MainTex, float4(input.texCoord.xy, 0, 0));
output.color *= _OutlineColor;