## Sunday, July 11, 2010

What is convolution ? Simply speaking convolution is weighted sum of pixel values around target pixel. If those weights are put into vector W and pixel values around x,y are put in vector P, then in linear algebra terms convolution at x,y is nothing more than dot product of vectors W,P. More precisely:

Convolution(x,y) = (W*P)/denominator + offset

This W vector is called kernel. (Also formula can be re-written for matrix case, in that case W and P would be matrix). So by using different kernels we could achieve different image convolution effects. Several common convolution kernels:

Gaussian Blur kernel
`1., 2., 1.,2., 4., 2.,1., 2., 1.`

Sharpness kernel
`-1., -1., -1.,-1.,  9., -1.,-1., -1., -1.`

Edge detection kernel
`-1./8., -1./8., -1./8.,-1./8.,  1.,    -1./8.,-1./8., -1./8., -1./8.`

Emboss kernel
`2.,  0.,  0.,0., -1.,  0.,0.,  0., -1.`

Now, GLSL pixel shader code which performs convolution with all these kernels =>

`#version 150uniform sampler2D Texture0;vec4 get_pixel(in vec2 coords, in float dx, in float dy) {    return texture2D(Texture0,coords + vec2(dx, dy));}float Convolve(in float[9] kernel, in float[9] matrix,                in float denom, in float offset) {   float res = 0.0;   for (int i=0; i<9; i++) {      res += kernel[i]*matrix[i];   }   return clamp(res/denom + offset,0.0,1.0);}float[9] GetData(in int channel) {   float dxtex = 1.0 / float(textureSize(Texture0,0));     float dytex = 1.0 / float(textureSize(Texture0,0));   float[9] mat;   int k = -1;   for (int i=-1; i<2; i++) {         for(int j=-1; j<2; j++) {             k++;             mat[k] = get_pixel(gl_TexCoord[0].xy,float(i)*dxtex,                            float(j)*dytex)[channel];      }   }   return mat;}float[9] GetMean(in float[9] matr, in float[9] matg, in float[9] matb) {   float[9] mat;   for (int i=0; i<9; i++) {      mat[i] = (matr[i]+matg[i]+matb[i])/3.;   }   return mat;}void main(void){   float[9] kerEmboss = float[] (2.,0.,0.,                                 0., -1., 0.,                                 0., 0., -1.);   float[9] kerSharpness = float[] (-1.,-1.,-1.,                                    -1., 9., -1.,                                    -1., -1., -1.);   float[9] kerGausBlur = float[]  (1.,2.,1.,                                    2., 4., 2.,                                    1., 2., 1.);   float[9] kerEdgeDetect = float[] (-1./8.,-1./8.,-1./8.,                                     -1./8., 1., -1./8.,                                     -1./8., -1./8., -1./8.);   float matr[9] = GetData(0);   float matg[9] = GetData(1);   float matb[9] = GetData(2);   float mata[9] = GetMean(matr,matg,matb);   // Sharpness kernel   //gl_FragColor = vec4(Convolve(kerSharpness,matr,1.,0.),   //                    Convolve(kerSharpness,matg,1.,0.),   //                    Convolve(kerSharpness,matb,1.,0.),1.0);   // Gaussian blur kernel   //gl_FragColor = vec4(Convolve(kerGausBlur,matr,16.,0.),   //                    Convolve(kerGausBlur,matg,16.,0.),   //                    Convolve(kerGausBlur,matb,16.,0.),1.0);   // Edge Detection kernel   //gl_FragColor = vec4(Convolve(kerEdgeDetect,mata,0.1,0.),   //                    Convolve(kerEdgeDetect,mata,0.1,0.),   //                    Convolve(kerEdgeDetect,mata,0.1,0.),1.0);   // Emboss kernel   gl_FragColor = vec4(Convolve(kerEmboss,mata,1.,1./2.),                       Convolve(kerEmboss,mata,1.,1./2.),                       Convolve(kerEmboss,mata,1.,1./2.),1.0);}`

What's left ? Results of course :)
Here is original image

convolution with Gaussian Blur kernel

convolution with Sharpness kernel

convolution with Edge detection kernel

convolution with Emboss kernel

1. awesome, thanks a lot!

2. I get an Syntax error : Array size must appear after variable name !!
Thank you.

3. I worked with shaders some time ago. If you run this with GLSLES (on ios for example) - it can be that glsl es version don't supports array initialization. So in that case statements like:
float[9] kerEmboss = float[] (2.,0.,0.,
0., -1., 0.,
0., 0., -1.);
should be rewritten like:
float[9] kerEmboss;
kerEmboss[0]= 2.0;
kerEmboss[1]= 0.0;
kerEmboss[2]= 0.0;
kerEmboss[3]= 0.0;
kerEmboss[4]= -1.0;
kerEmboss[5]= 0.0;
kerEmboss[6]= 0.0;
kerEmboss[7]= 0.0;
kerEmboss[8]= -1.0;
Hope that helps ! Check this link also - it may help:
http://stackoverflow.com/questions/28435438/how-to-write-const-array-in-glsl-es

Comment will be posted after comment moderation.