Image Inpainting

An updated version of this post is available here.

Image Inpainting is the art of filling in lost portions of an image using background details in a visually plausible way. I came across this topic while searching for projects in the field of image processing for sixth semester. The moment I saw a few sample pictures, I decided that I was gonna work on it.
Here are a few pics of what we achieved in the end using our software:
Source Image for Inpainting Result after performing Inpainting

As is quite clear from these images, a good and fast algorithm for Image Inpainting can be really useful. I worked with Sapan to develop the software. We started with a traditional algorithm but finally ended up making several changes to it in order to improve the results.

Here’s a brief decription of the inpainting algorithm.

  1. Divide the marked region into patches.
  2. Assign priorities to the patches (Using Data and Confidence Terms)
  3. Choose patch with maximum priority and find a patch from background region that is most similar to the chosen patch
  4. Replace the patch with the patch from background and update data and confidence values.

This is a very brief overview of the inpainting algorithm. For more details, you may feel free to contact me or go through the research paper that we authored.

For more information about the implementation, visit our project page on Sourceforge. I hope that you would enjoy using it. It is still a very early release and we hope to improve on it soon. Looking forward to your participation on the project as well.

See the software in action in this video.

Stay tuned for my next post. Meanwhile, tell me your suggestions/comments about my work.

Advertisements

19 responses to “Image Inpainting

  1. rajesh jadav March 3, 2011 at 11:47 AM

    Dear sir,

    I also want to do my study in the field of image inpainting, so i need your help how i go through it. please guide me

  2. soncan May 30, 2011 at 10:50 AM

    Hi Pulkit,

    Did you try this algorithm with small region like 10×10 pixels?
    I got problem with my code in small with high contrast

    • Pulkit Goyal May 30, 2011 at 12:55 PM

      Hi,

      By region of 10×10 pixels, do you mean the patch size? I have tested the algorithm with a patch size of 9×9 pixels as well as 3×3 pixels.
      Can you mail me the image that you are working with?

  3. priya January 6, 2012 at 3:59 PM

    hi sir
    i am in final year
    i am doing project on image inpainting
    we do our project in java
    i hav some doubt
    we hav take ieee papers for references
    in this there is some many equations
    i cant understand it carefully plz give some idea to understand it
    plz give some idea to find the pixels in the image except in the masked region

    • Pulkit Goyal January 7, 2012 at 4:13 AM

      Hello,

      For this you need to iterate through your image looking for patches that best correspond to the patch in question. This can be done by using some error measure. We use Mean Squared Error to calculate patch similarity. I hope this answers your question.

      • priya January 7, 2012 at 11:20 AM

        hi
        thank you for your reply
        we studied mean squared error algorithm but we cant understand clearly
        plz explain for me clearly

  4. priya January 7, 2012 at 12:24 PM

    hi
    how r u
    we hav just identified the masked region
    we hav no idea to proceed further so plz guide me
    but we hav no idea of how to map the pixels in that area
    we hav some doubt on corespondence map

    • Pulkit Goyal January 7, 2012 at 2:56 PM

      After defining the mask region, you need to find the boundary of the mask. This can be done by using any traditional gradient method like the forward difference method. Once you have the boundary of the masked region, you can then calculate the priorities of all the patches on the boundary of the unknown region and then try to fill the highest priority patch. This can be done by finding the patch from the background that has the lowest mean squared error as compared to the known pixels of current patch. The mean squared error is just the sum of squares of differences between corresponding pixel values between the source patch and the target patch. I hope its clear to you now.

  5. priya January 10, 2012 at 11:03 AM

    hi
    thank u for your guidance
    we understand it and we are now going implement it

  6. priya January 11, 2012 at 9:50 AM

    plzz
    send the code in java

  7. priya January 11, 2012 at 2:16 PM

    pls send the code in java!!!!!!!!!!!sir

  8. priya January 11, 2012 at 2:18 PM

    thank u sir
    but we cant download the source code from the link

  9. priya January 13, 2012 at 1:45 PM

    thank u so much!!!!!
    sir!!!!!!!!!!!!!!!!!!!!!1

  10. priya January 13, 2012 at 2:29 PM

    thank you so much brother!!!!!!!!!!!!!!!!!!
    we hav execute the code and get the answers
    thank you
    we are very happy now
    thank qqqqqqqqqqqqqqqqqqqq!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!1

  11. user1 January 30, 2012 at 10:29 AM

    Message Deleted!

    • Pulkit Goyal February 1, 2012 at 7:05 PM

      Gradient is used in data term to give importance to the structural features in the image. The exemplar technique is looking at the image in patches and then filling the patch with the maximum priority. This refers to the patch that we have a high confidence in filling (because we know many of its pixels already). I would recommend you to read this paper thoroughly?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s