Your SlideShare is downloading. ×
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply



Published on

This paper presents the super-resolution algorithm for text …

This paper presents the super-resolution algorithm for text
images, capable to resize small text image to a bigger one
with minimal loose of quality. The algorithm basic flow is
divided in 3 main tasks. First task is to generate the
dictionary patterns from a big resolution sample text image.
The second task is to obtain the nearest similar dictionaries
from the low-resolution image and replace them. The last
task optimizes the new resized big-resolution image by
reducing the noise and improve quality of the characters
presuming that the single characters are represented

Published in: Technology, Art & Photos
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. SUPER-RESOLUTION OF IMAGES BASED ON LEARNED DICTIONARY PATTERNS Peter Karlovšek, Sandi Gec Faculty of computer and information science Tržaška cesta 25, 1001 Ljubljana, Slovenia e-mail: ABSTRACT KEYWORDSThis paper presents the super-resolution algorithm for text - matriximages, capable to resize small text image to a bigger one - dictionarywith minimal loose of quality. The algorithm basic flow is - cubic spline interpolationdivided in 3 main tasks. First task is to generate the - patterndictionary patterns from a big resolution sample text image. - resolutionThe second task is to obtain the nearest similar dictionariesfrom the low-resolution image and replace them. The lasttask optimizes the new resized big-resolution image byreducing the noise and improve quality of the characterspresuming that the single characters are representedcontinuity.1. INTRODUCTIONThe aim of super-resolution system is to resize text imagefrom small resolution to big resolution text image usinglearned dictionary patterns. In this way we want to obtainbetter results than well known cubic spline interpolation [3]used in most raster scale software. In this way we get sharpercharacters of the text. Our resize system can be used toresize small resolution scanned text, resize scanned oldmanuscripts and similar texts. Similarly approaches wereused to resize small-resolution color images to high-resolution and are presented in the following chapter Relatedwork.The first step of our system is to generate dictionary patterns.Precondition of this step is that we have a high resolutionsample text. By generation the dictionary patterns we Figure 1: Presentation of the main flow of the super-preserve the information about font characteristics. resolution system.In the second step we generate the first version of the resizedhigh resolution text. We use the dictionary patterns from the 2. RELATED WORKprevious step and replace the low-resolution pattern with theproper high resolution pattern. There are presented different Related approaches with our work are two methods: (i) Theapproaches to find the proper high-resolution pattern. classical multi-image super-resolution, (ii) example basedIn the third step we present the optimizations to improve the super resolution, and (iii) super-resolution from a singlequality of the resized high-resolution image text. We have to image[1].remove the noise from the image, replace the bad chosen Super-resolution from a single image (iii) works assume thatpatterns with the better ones. images contain repetitive visual content of small size. WeBy implementing all the three steps shown in figure 1 we search patches until we find at least two similar patches inevaluate some results using different types of fonts and different scales. The example of finding patches from theconfront the results with cubic spline interpolation scaling. image is shown in figure 2. This approach is effective if only several high-resolution parent patches matches low- resolution patches. That way of solving super-resolution is limited and not suitable for solving our image text resizing because our image does not contain many related visual
  • 2. content because our text images contain characters with the we generate for example 200.000 different dictionarysame font size. patterns the time increases significantly. Before generating dictionary patterns we resize high- resolution image to a low-resolution. Then we resize the low-resolution image to a big-resolution using cubic spline interpolation. After that we have to text images of the same size: first image is high-resolution text image, second one is the low quality high-resolution obtained by resizing from low to high. By generating non duplicated dictionary patterns from high- resolution image for each dictionary pattern we also save the pattern from low quality high-resolution image. The result of this step are 2 dictionaries each size of 100.000 non duplicated matrixes of size 7x7. The connection between dictionaries is the representation of pattern in high-resolution high quality and high-resolution low quality. Figure2: Patch recurrence within an across scales of a single image Figure 3: First 4 images represents high-resolution patternsThe most relative to our paper is (ii) example based super and next 4 images represents their mappings in low-resolution which uses similar approach about building resolution.patterns of our first step. Following steps are different toours because we are not working on RGB scale images. Our 4. SUPER-RESOLUTION ALGORITHMimages are more specific, only grayscale text images. The Once the dictionary patterns are generated have the base ofwhole process about obtaining a high-resolution text images our super-resolution algorithm. The input of the step is ais presented in following chapters. low-resolution test image that we want to resize with the super-resolution algorithm. First we simply resize our low-3. GENERATING DICTIONARY PATTERNS resolution test image with cubic spline interpolation to a highOur process begins by generating dictionary patterns. The resolution. We obtain a low quality text image of desiredwhole application is implemented in Matlab. Precondition of high-resolution size.this step is a high-resolution text image which must contain After the proper size of the image is prepared we improverandom characters preferably most of the alphabet the quality of our test image by replacing the image matrixescharacters. If we use high-resolution text image with not of size 7x7. We shift from left to right and up to down byenough different characters the result will be homogenous iteration of 7 and replace the patterns with proper one fromgenerated dictionary patterns followed with bad super- the high-resolution image.resolution resized image from low-resolution. So we Our first approach was to calculate histogram from the testgenerate heterogeneous dictionary patterns matrixes size of image pattern and compare to the histograms of the low-7x7 pixels randomly from the image. We tested the resolution dictionary patterns. The proper high-resolutiondifferences between number of generated dictionary patterns pattern is obtained by using one of the distance methods L2and concluded that the recommended number is 100.000 [5], Chi-squared [6], Hellinger [7] and section-distance[8].different dictionary patterns. If we generate less than This approach works but better results are shown by using100.000 dictionary patterns the quality of the super- the comparison between vectors and not histograms.resolution text images is visibly worse, on the other side if Our pattern vectors of size 1x49 are build from pattern matrixes 7x7. We calculate the distance between vector
  • 3. pattern from our test image and every dictionary pattern elements of the edge does not contain any gray or blackusing formula for mean square error [4] in our case mean element that means that this pattern candidate does notsquare distance (MSD). Every dictionary pattern must be satisfy the condition of continuousness. That optimizationfirst reshaped from matrix 7x7 to vector 1x49 before approach is presented in figure 5.calculation of MSD defined by the following equation:MSD =An example shows 3 patterns and their best 10 nearestcandidates of high-resolution patterns and is presented infigure 4. Figure 5: Continous condition is reached if at least one element of matrix marked with red contains gray color value. 6. RESULTS In this chapter we present the results of paper work. The results are presented on two different fonts and different test low-resolution text images. These test low-resolution images are resized with cubic spline interpolation. Selected fonts tested with our super-resolution system are: Times new Figure 4: Test patterns (orig) and their nearest 10 roman and Brush Script MT.Figure 6 and figure 7 shows representations of high-resolution patterns. high resolution text images used to generate dictionary patterns with different fonts (1. step).That first phase of super-resolution algorithm replaces all of7x7 matrix patterns with a high-resolution ones. We must becareful when using MSD that we calculate the distancebetween test low-resolution pattern and low-resolutiondictionary patterns. When replacing the best matches welook how low-resolution pattern is represented in high-resolution pattern. This is why we generated 2 dictionarypatterns in the previous step.The following chapter describes optimization approaches toimprove the quality of the super-resolution image.5. OPTIMIZATION APPROACHESLast step of our super-resolution system is the optimizationstep to improve the quality of the super-resolution testimage and remove the noise. We tried different optimizationapproaches and two of them showed us best results. Figure 6: High-resolution sample using font Times newThe first approach is to ignore the 7x7 patterns that does not roman.contain any black dot element represented in grayscale withvalue 255. We replace that kind of patterns with whitematrix of 7x7 elements. In this way we get rid of gray noiseand improve the sharpness of our super-resolution image.Another good optimization approach is to presume that allof the characters are continuous. By presuming that wecheck edge of the 7x7 matrix of any candidate pattern. If
  • 4. Figure 13: High-resolution text image obtained from cubic spline interpolation from figure 12. Figure 14: High-resolution text image obtained from our super-resolution system from figure 12. Figure 7: High-resolution sample using font Brush Script MT. Figure 15: Ideal high-resolution text image resized fromInputs for testing our super-resolution algorithm are text figure 12.images shown in figure 8 and figure 12. Final results madewith 2. Step and optimized in 3. step are shown in figure 10 Our result with font Times new roman (figure 10) in confrontand figure 14. For objective result confrontation we use of cubic spline interpolation (figure 9) is sharper. The linesimage resize method cubic spline interpolation shown in of characters have been recovered quite well. Text isfigure 9 and figure 13. Ideal results of high-resolution text readable but includes some noise due non perfect meanimages are shown in figure 11 and figure 15. square distance replacement. Text from cubic spline interpolation (figure 9) contains many unnecessary gray around every character. Our optimization approaches satisfactorily reduces the gray noise from the image and limits that characters are shown cut (uses presuming that allFigure 8: Small-resolution text image prepared to be resized characters are continous). to high-resolution text image. Font of the text is Times new Similar results are obtained using font Brush Script MT roman. (figure 14). All the characters are readable but our super- resolution system returns better results from fonts with straighter lines. In that case we could use some more optimization approaches to fill the missing (white colored) characters data. Some of serif character details, for example from figure 12 in the last character “E” and from figure 10 in Figure 9:High-resolution text image obtained from cubic the first character “P”, are better recovered than cubic spline spline interpolation from figure 8. interpolation. 7. CONCLUSION We developed a super-resolution system that returns readable high-resolution text. The system does not recover Figure 10:High-resolution text image obtained from our the ideal characters quality but it is good enough for testing super-resolution system from figure 8. with some Optical character recognition (OCR) systems [9]. Better results could be reached with additional optimization approaches but in that case the possible problem could present in time complexity. This system is not mentioned to be used for real time Figure 11: Ideal high-resolution text image resized from processing in real time systems because of current high time figure 8. complexity T(N) = n3. The point of usage could be for example to recover tiny character quality from scans of old manuscripts and other damaged old texts. Another Figure 12: Small-resolution text image prepared to be interesting usage approach could be to recover tiny text resized to high-resolution text image. Font of the text is without knowing the font. In that case we must generate a Brush Script MT. high amount of dictionary patterns from known fonts and test our super-resolution system on each dictionary pattern.
  • 5. From best readable result we could determine the originalfont.References [1] Glasner, D., Bagon, S., Irani, M.: Super-Resolution from a Single Image, In: Proc. of ICCV (2009) [2] Freeman, W., Jones, T., Pasztor, E.: Example- based super-resolution. IEEE Computer Graphics and Applications, 56-65 (2002) [3] ion (January 2012) [4] http://www.math- urve_to_data/root_mean_squared_error.htm (January 2012) [5] ce (January 2012) [6] edStatistic.html (January 2012) [7] e (January 2012) [8] /Distance_Between_2000.pdf (January 2012) [9] r_recognition (January 2012)