Optimization Methods in Digital Halftoning

Transcription

1 Optimization Methods in Digital Halftoning Team Digital Halftoning Thomas Bredl Daniela Fußeder Erik Johansson Lisa Kehrer Katharina Wohlgemuth

2

3 Is my Printer a Maths Genius? Thomas Bredl Daniela Fußeder Erik Johansson Lisa Kehrer Katharina Wohlgemuth July 2009 Abstract Printers are common household devices today. But although we can create high gloss photos at home, printing grey halftoned areas is still a challenging optimization problem. In a grey scaled printout one may notice some patterns, so called textures. They occur when converting the continuous image to a binary output and are, of course, unwanted. But although there already exist mathematical approaches to prevent such patterns, technical restrictions of laser printers interfere with these methods: Printer requirements counteract visually acceptable printing results. Furthermore one single method is required to work well for all shades of grey simultaneously. It is not known if there exists an optimal solution at all. This is to give an approach to solve the problem through discrete optimization. The best results are achieved through the maximization of distances of the printed dots within the technical restraints. With this ansatz we can present a high average of satisfying results for greyscales. In addition we show several mathematical quality criteria to enable a more scientific comparison of methods, which is no longer based on eyesight. 1 Where is the problem? My printer is a wizard. In our modern computer times we are used to seeing spectacular animations and transformations on the screens of our pc. We are not awed anymore by the simple fact that we can print any picture of us and that it really does look like us and not like John-the-big-Blur. But this is not just a simple process, indeed it requires mathematics, physics, informatics, chemistry - you name the game. Have you ever thought about how exactly the printer knows where the colour has to go on your sheet of paper?how the machine is able to transform electronic information displayed on your screen into a touchable real world object? If you think back to the typewriting time, it was easy to see the mechanics, but nowadays it works 1

4 like everything else in the IT world namely with numbers - nowadays printing takes place in a black box. In the wizarding world of Harry Potter it would have been a subject at Hogwarts called Transfiguration. My printer is a secret agent. In this article we want to give you a glimpse into this world of numbers so that it does no longer seem like magic. Let us assume you want to describe your latest photography of your dog to a friend of yours via telephone. You might start out by telling him, that in the middle of the picture there is the dog in brown colours standing next to a tree with a house in the background. This description might give your friend an idea of it but he will hardly be able to reconstruct it exactly. For instance he neither knows the exact position of the dog nor the kind of brown of his fur. You see, if we want this experiment to work we ought to find a better method to hand down the information. If we numbered each and every colour, and you and your friend had both a table of all colours and their values at home to look it up, we could talk about the exact shade of any brown we want, so the dogs tail might be light brown or has the colour 217. Of course, we would have to agree on some positioning coordinates too, for instance chop the picture into several small boxes (which will then be pixels) and number them. Try it out: If we agreed on the colour code on the left, what picture the numbers on the right would give you? Colour Code of Colour Table 1: Solution to the quiz at the end of the article. Now the same problem of telling your friend about your favourite picture occurs if you want to bring your printer to put it on paper. And the same method of coding the photos works fine with it too. In a way we just exchange the digital picture by one which is encoded as numbers and our printer has to decode it in its own language. For simplicity let us only look on greytones. Because of technical reasons we have a range of 256 of them, but that should be sufficient for any grey scaled image. 2

5 My printer is a potions master. Telling your printer the code of the photography is only the slightest problem in the whole process. Have you ever bought 256 cartridges of ink for your printer? Probably you have not, because otherwise your printer would be as big as your washing machine. Usually one buys only one kind of black ink. So how does the printer get all the other colours (in our simplified case the 254 shades between black and white)? This is a good time to get some insight into the way a printer works: An ordinary printer can either print a pixel black or leave it white. Since we do not want to print only black and white pictures, the printer has to place black and white dots next to each other patterned in such a way that the area will look like a smooth greytone for the human eye. If you look at your printout with a magnifying glass, you will still be able to see the individual dots. Figure 1: left: grey 200 as it is displayed on your screen, right: grey 200 as your printer would print it in high definiton Figure 2: left: grey on your screen, right: grey high definition of printout My printer is a bull in the china shop. How does a laser printer actually work? Technically it is like this: There is a photoconductor, which is a roll where toner particles are placed. When it turns it prints the particles on the paper, just like a stamp. The photoconductor on the other hand gets the particles from the jump roller: The printer can ionise both particles and the conductor. Wherever we want to have a pixel blackened, we put charge on exactly that spot on the conductor. Then the toner particle, which is charged differently is drawn from the jump roller to that place directly. You can imagine it to be like a magnet sitting on the roll, attracting the toner particle. The problem is, that because of physical constraints, our mechanical artist fails to print one and only one pixel at an exact position. The spirit 3

6 Figure 3: Schematic setup of a laser printer. is willing but the flesh is weak - it needs to cluster black pixels in order to be able to print them. My printer is a designer. Now the question arrises how to position the black and white dots for each greytone. Since we learned that there has to be a certain quantity of electrons and hence some black pixels there is yet another problem at our hands. That is to say if we place same coloured dots next to each other, we might get patterns! In a smooth digital greytone we simply do not want to have any lines, circles and so on in the print version. You see, our printer has to carefully design such a distribution. This is why we design one model which tells the printer exactly how it should transform each greytone. We then compare the encoded picture with our model. Since we want to use the model for any kind of size of the original image, we decide to split the picture into tiny parts, let us assume that these parts control the output of 4 times 4 pixels. Take any grey scaled image of the size you like, for instance a square of 3.2 mm 2. It disolves into 12 x 12 pixel. Then we tile the image so that our model fits in. In this case we could place our model 9 times on the image. Figure 4: Fit the model on the image. 4

7 Now we have several smaller images than before. In each of the small boxes the value of the pixel is enlisted. (Just as we did with the picture in table 1). With grey scaled images it is a bit boring of course, because in each and every model every pixel value is set on the number of that shade of grey. Namely like that (in one of the small parts): Table 2: Encoded grey scaled image. By the way, the scientific name of our model is threshold-matrix, which sounds great, but after all it is just a name! For demonstration purposes we may assume that our model looks like that: Table 3: Threshold-matrix for testing. Now we start comparing the matrix to the image. Wherever there is a number in the threshold-matrix which is less or equal to the number in our coded picture, we decide to print a black pixel Table 4: Threshold-matrix for testing (red means, that there the value of the Matrix is less than or equal to the number of the greytone). We now create a new table where we put down a one for yes please, print this specific pixel black, and zero if we want to leave that pixel white. What we get is a binary 1 matrix. Here the binary matrix would look like this: 1 because it only holds ones and zeros 5

8 Binary matrix. Printout, zoomed in. You really came a long way, but the true work is only starting. We will need some elbow grease now. This is where the Magic, oops, Mathematics, is coming in. 2 The Maths Genius. As we worked out in the previous section we need to find a model for our printer so that it can transform any smooth grey scaled image into a binary image. We gave the model a nice scientific expression, namely threshold matrix. With it, we can split any grey scaled image into smaller parts and compare each part to the matrix. By doing so we decide where to put black pixels and where to leave them white. In our example we can imagine that it could result in some patterns if we repeatedly placed the binary-image next to each other (to get a bigger area of grey). It will not give you a smooth grey surface resembling the input image. In the mathematical part of this article, we would like to introduce some ideas on how to find a thresholdmatrix which produces nice grey coloured surfaces. But what exactly do we mean by nice? 2.1 Quality criteria. As you look closer at a grey surface your home office printer produced, you might find little black or white crosses or dots in the apparently smooth grey surface. You might even find lines that pervade your whole greytone. These kinds of textures, we would like to avoid. But how do they occur in the first place? Deciding where to place our black and white dots when mixing greytones, it happens, that we place two black dots directly next to each other. If we do this continually, we get a black dot, cross or line in our grey surface. See for instance if we arranged the following binary-matrix nine times how that would look like? As you can see, there will be some vertical lining in our printed version! This leads to our first measurement of the niceness of a grey area. Roundness Criterion As we want to avoid structures like lines or crosses, we want to choose our threshold-matrix in a way that we get clusters which are as round as possible. To verify the roundness of a cluster, we have to check three things: 6

9 Example of a bad binary matrix. Its output. Table 5: Output of 9 accumulated matrices. Symmetry Convexity Inner Pixel Circle On Symmetry: First, we remember that a circle has an infinite number of symmetry axes through the centre. Therefore, if our cluster has a nice round shape, it should also have many symmetry axes. Thus our first criterion on roundness is to maximize the number of symmetry axes. Figure 5: Example of symmetries of a circle. 7

10 Figure 6: One axis of symmetry on the left cluster, opposed to the unsymmetrical cluster on the right. Put into mathematical language: max S where S is the number of symmetry axes. We also want to check, if the cluster is symmetric to a point. If so, we set P = 1, otherwise, we set P = 0 where P is our variable which measures the point symmetry. Obviously we would like to have P equal one, so we can just as well write max P, because the maximum of 0 and 1 is 1. On Convexity: Secondly, we define the shape a rubber band will get when you tightly tie it around a given object. This is an informal definition of the convex hull of an object. We then write conv(c) when we refer to the convex hull of an object C. One can easily see, that the convex hull of a circle is the circle itself. So we want our cluster to have a convex hull as small as possible. Thus, when the difference between the area enclosed by the convex hull and the area enclosed by the considered cluster, is small. Therefore our second criterion on roundness is to minimize this difference. Figure 7: Red rubber band around pixel assembly. Again, writing it as a nice formula would give us: where C is our cluster of pixel. M C = area(conv(c)) area(c), min M C On Inner Pixel Circle: Third, a inner pixel circle of an object C is defined as the smallest circle with a centre in the cluster, which either completely covers the edges of a pixel or does at least touch them. We write ipcircle(c) for the inner pixel circle of an object C. The cluster circle of a circle is just the circle itself. But for a cluster of pixels, call it C, we certainly will get a difference between the area covered by its cluster circle and the one covered by the cluster. Again, we want to minimize this difference. M K := area(ipcircle(c)) area(c), min M K. 8

11 Figure 8: Example of an inner pixel circle. Summed up: Just to make sure we will not forget any of these measures, we just sum up the short notations again: C is our cluster of pixels. Symmetry: max S, where S is the number of symmetry axes max P, where P = 1 if the cluster has a point symmetry, = 0 if not. Convexity: M C = area(conv(c)) area(c) and min M C Inner Pixel Circle: M K := area(ipcircle(c)) area(c), min M K It will take some time to check the threshold-matrix threshold-matrix for all of these. It would be so much faster to just have one criterion. And we can actually have that. We just combine all of our four criteria to one super measure M. We could for instance just add up all the results of the criteria we get for one cluster. The problem is that the cluster circle criterion and the convex hull criterion produce a figure which is measured in units of area and the symmetry axes criterion produces a natural number. If we just added up the numbers, it would be like comparing apples and oranges. Therefore we must find a way to compare the criteria in a way that the different semantics do not bother us. How about adding up the cluster circle criterion and the convex hull criterion and divide it by the number of symmetry axes. As we cannot divide by zero but can have clusters with no symmetry at all, we add one in the denominator and get: M := min M C + M K P + S + 1 This was a lot of work, but now we have a nice roundness criterion which takes account of all our considerations. We just need to confirm that it does the same as each criterion does for itself. If we minimize M, the numerator M C + M K is getting smaller, which means we add min M C and min M K which is just what the convexity and inner pixel circle criterion require. Hence we are in accordance here. To make the fraction in M smaller, we have to increase the denominator, that is, we want to have P + S + 1 as big as possible, but that leads to maximizing P + S and this is nothing else than hoping for as much symmetry as we can get. 9

12 2.2 Of finding the threshold-matrix. Now that we have talked so much about what would be nice to have, we actually want to find our threshold-matrix that turns each grey scaled image into a binary image that the printer can process. There are many different approaches in mathematics, for example methods inspired by stochastic or calculus. We would like to show you an idea using geometry and optimization. As we have seen before we have to obey two conditions. One is, that if we cluster, we want to cluster them densely, leaving as little space as possible in between, so that they appear as round as possible. On the other hand, we do not want to grow too many clusters on one spot, since then we could get horrible textures. This time we do not come from an explicit colour to the threshold-matrix, now we do it the other way around. That is, we are starting with a matrix without any entries and try to fill it up. Since we can recognize textures only by arranging more of those matrices, we do exactly that. We place the minimum set of matrices, 9 at a time, next to each other, with the one in the middle as our reference. In the picture below you can see such an ensemble, the yellow marked matrix is the middle one, where all the changes will take place first. We have inserted randomly a starting pixel which is given number 1 (it is the red marked one), that means, it is the first one set to black for any grey scale. So, if we want to transform a grey area valued 100, it definitly gets an entry 1 in the binary matrix for yes, this pixel at that position must be printed black. We multiply the dot so that each of the matrices look alike. Now we are looking for the pixel which turns black after the first one has. If you had a greytone 1, then only pixel number 1 would flip to black but if you put in grey scale 2 both would turn on. So, which entry should we turn to 2, so that we move within our constraints. The method we are applying works this way: We need to find a position for our new black dot in the reference matrix, which has a maximum distance from all already existing dots in our three by three arrangement of the threshold-matrices. So to speak we want to fit in a circle which is centred at the new dot and has the biggest possible radius. Now we choose the pixel which is the centre of the circle as our next black pixel and copy it into the surrounding matrices. 10

13 i) ii) Figure 9: i) Draw a maximum sized circle with a centre in the thresholdmatrix. ii) Duplicate the new found pixel If we repeated these steps several times we soon would have our matrix. But until now we did not give you the draw backs. But here they are: Our matrix is an 18 x 18 table. The smallest possible number would be 16, as 16 2 = 256 but usually an 18 x 18 matrix is used. We cannot chose arbitrarily. There is another set of restriction which we will carefully explain to you below. As promised we will show you further constraints now. We have to lay a grid over our matrix where the intersections represent so called raster centres. You can see it in Figure 10. Figure 10: 17 raster centres - within red circles The restriction now is, that we absolutely have to start in one of the centers, this, however, diminishes our choices. The next number has to lie also in one of them, until all 17 centres have entries. Our method still works - we just have to draw for each allowed position a circle as we have described above and find out, which one has the biggest radius. After all 17 centres are filled, we want to find the best distribution for the next 17 pixels. Now we cannot place them AT the intersection, but clearly we can place them AROUND, which is in accordance to our quality criteria we worked out in section 2.1. We only have to suppress the positions we have already found, 11

14 because otherwise there would be no biggest radius! This is the basic idea with which we work through all 18 x 18 possible positions. STOP you might cry out any time now, because, if you were very observant, we only have 256 different shadings of grey, but with the 18 x 18 threshold matrix we would get 68 more colours! Well, this eventually is something to our advantage: We are allowed to throw away 68 out comings that are not visually pleasing. Is not that something? We have printed our final threshold-matrix that we have obtained through the approach we just described. At first sight it is just a bunch of numbers, but we hope that we could show you, what is behind it. At least we hope you got an idea about how mathematics can be used and how it affects our daily life. You may not encounter it when you go to the mall or cook dinner. But be aware, since this example shows you that there is maths just everywhere, there might be someone pointing out the maths at the shopping mall after all! Table 6: Final threshold-matrix. Figure 11: Solution to the test in Table 1. 12

15

16