Unity3D Chroma Key Shader

I thought that finding a shader that makes an area of a texture of a certain colour transparent would be easy. It turns out that finding an efficient and free chroma key shader required a little bit more digging.

The best implementation I found was on a blog called Pilcrow Pipe, but it was written in OpenGL shader language, not Unity3D’s ShaderLab language which is more universal and allows you to change parameters such as key colour and tolerance through the Unity GUI. So without further ado, here is the ShaderLab port of the chroma key shader.

// Adapted from http://pilcrowpipe.blogspot.jp/2013/03/chromakeyingtransparentbackground.html

Shader Custom/ChromaKey {

    Properties {
        _MainTex (Base (RGB)2D) = white {}
        _thresh (ThresholdRange (016)) = 0.8
        _slope (SlopeRange (01)) = 0.2
        _keyingColor (Key ColourColor) = (1,1,1,1)
    }
    
    SubShader {
        Tags {Queue=Transparent IgnoreProjector=True RenderType=Transparent}
        LOD 100
        
        Lighting Off
        ZWrite Off
        AlphaTest Off
        Blend SrcAlpha OneMinusSrcAlpha 
        
        Pass {
            CGPROGRAM
                #pragma vertex vert_img
                #pragma fragment frag
                #pragma fragmentoption ARB_precision_hint_fastest

                sampler2D _MainTex;
                float3 _keyingColor;
                float _thresh// 0.8
                float _slope// 0.2

                #include “UnityCG.cginc

                float4 frag(v2f_img i) : COLOR {
                    float3 input_color = tex2D(_MainTexi.uv).rgb;
                    float d = abs(length(abs(_keyingColor.rgb – input_color.rgb)));
                    float edge0 = _thresh * (1.0 – _slope);
                    float alpha = smoothstep(edge0_threshd);
                    return float4(input_coloralpha);
                }
            ENDCG
        }
    } 
    
    FallBack Unlit
}

I hope that somebody finds this useful.

Again, all credit for the logic behind the shader goes to this blog post.

Advertisements

16 thoughts on “Unity3D Chroma Key Shader

  1. Hi noxharmonium

    Thanks for putting this code together. I have been trying to figure this out for months and stumbled upon your post. Can you explain how I attach this to a video in Unity3D? I am using this with Vuforia.

    Thanks for any help you can provide.

    -Phil

    Reply
    • You can get a plugin to play videos from the Asset Store. Qualcomm’s Vuforia has a sample which plays videos when you look at a marker that you can extract the video player from as well. The video player plugin will work by updating the texture on a material attached to a plane. You can just set the shader on that material to this shader.

      Reply
      • Hi noxharmonium – great site! Thanks for the response.

        So are you saying there is a pluging on the UNITY Asset store that allows for keying the background out of a green screen video? Can you recommend which one you suggest? Thanks!

      • What platform are you building for? We had to target iOS and Android so we had to find a plugin that worked on both but if you are only targeting one platform it will be easier.

  2. Primarily for iOS and we are using Unity free not the paid pro version yet. We are still building some demos at this point in time. Thanks. > > >

    Reply
  3. Thanks so much for the code, it is working really well!

    Could you explain a little more about the slope and thresh? It seems to make some part of the video that I’m applying this to semi-transparent. It doesn’t seem like the semi-transparent part has a color that is close to the green that i’m keying out. Any idea how I can avoid that from happening?

    Thanks so much!

    Reply
    • The shader works by finding how different the color of each pixel is compared to the keying colour. Usually you want a low threshold which means that only pixels with colours that slightly different to the keying colour are made transparent. As you increase the threshold the similarity required to make the pixel transparent reduces until full threshold where every pixel is made transparent.

      If the slope parameter is set to 0 than the pixel is either transparent or not transparent but as you increase the slope the pixels alpha value will transition smoothly from opaque to transparent as the color becomes more similar to the keying colour.

      You may run into problems with parts of your footage that you want to be opaque becoming transparent due to things like the green screen colour spilling onto your subject. Professional video editing tools have very complex chroma keying tools with features that you couldn’t squeeze into a high performance shader. The footage that I used with this shader I had to run through a video editing program to use spill suppression, clean up the green screen to a solid colour and fix up the edges of the subject of the footage.

      Hope that helps!

      Reply
  4. Thanks for clearing things up! I tried keying out the background with AE and added a solid green color to it afterwards. It seemed to work ok but there was green glowing edge in some part of the object. I presume that I have created a semi-alpha edge on the object when editing that caused the green edge. Will play around with the keying and see if I can get rid of it.

    I have also read elsewhere that a split alpha method is also possible in achieving the transparent movie effect, that has the original footage and an illuminance map on the same video. Would you by chance know where I could obtain a shader as such? I was hoping that it would work along with the Vuforia Video Player prefab. Thanks so much!

    Reply
    • There is a Unity Answers post that has two versions of the split alpha shader. They are actually quite simple, except that you need to synchronise two videos, one with the RGB and one with the alpha as greyscale. The greyscale alpha video can even be a lower resolution than the RGB video to an extent depending on the shader. I’m not sure how to go about creating the greyscale alpha video, maybe you could do it in After Effects.

      If you are having problems with getting the two videos to synchronise than I think as better way would be to combine the RGB and greyscale alpha videos side by side into the same video and rewrite the shader to get the RGB from one side and alpha from the other. I’ve seen it done before but I can’t seem to find the shader online. I think the “Mobile Movie Texture” plugin comes with one.

      Reply
  5. Thanks for the heads up, I’ll take a look at the shaders in the posts at Unity Answers and see if I can get something working. The combined RGB and greyscale alpha was actually the solution that i was looking for, I have also bought the Mobile Movite Texture plugin in the past but unfortunately it only runs ogg video formats and not mp4 (and also the author of the plugin has seemed to ceased support…). I don’t really know much about shaders but I’ll test my luck and see if I can get any sense out of the links that you provided.

    Thanks so much again, wishing you a Happy New Year and a prosperous 2015 🙂

    Reply
  6. Hi noxharmonium.
    thanks for your code about transparent video. I want to implement this code to vuforia sample videoplayback project. I am a junior programmer on unity, so I have some problem when I implement this source code to the project. please check this screenshoot:
    https://drive.google.com/file/d/0B1PFzpt6cOEJb25QV0lNV0UybTg/view?usp=sharing

    if you are willing to help me , please tell me step by step how to implement this source code to unity project. Thanks

    Reply
    • I don’t spend much time in Unity these days, it is possible that Unity have updated their shader syntax or something since I wrote this. I will try to have a look this week to see if it is still working. If you have solved the issue please post it here!

      Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s