BU3 Project Proposal Group Members 1. Ms.Watcharaporn Sitsawangsopon ID: 5422791509 2. Ms. Maetawee Juladash ID: 5422772905 Advisor: Dr. Bunyarit Uyyanonvara (Associate Professor) School of Information, Computer and Communication Technology, Sirindhorn International Institute of Technology, Thammasat University Semester 1, Academic Year 2014 Date: December 15, 2014
Table of Contents 1 Introduction... 1 2 Background... 4 3 Objectives... 6 4 Outputs and Expected Benefits... 6 4.1 Outputs... 6 4.2 Benefits... 6 5 Literature Review... 7 6 Methodology... 8 6.1 Approach... 8 6.2 Tools and Techniques... 12 7 Project Schedule... 14 8 References... 15
Statement of Contribution By submitting this document, all students in the group agree that their contribution in the project so far, including the preparation of this document, is as follows: 1. Ms.Watcharaporn Sitsawangsopon ID:5422791509 50% 2. Ms. Maetawee Juladash ID:5422772905 50%
Introduction In the medical profession of a facial skin, many patients that come with several problems and skin lesion types, compare with the amount of doctors that less than the patients in many countries around the world. So the technology nowadays can help the medical profession and doctors at least to analyse the problem on each patient's face. This can decrease the situation of doctors and patients to be in face to face, help the doctors and patients that resident in different countries, and also decrease the problem of ratio between doctors and patients. The cause of the problems make use of a computer to aid in maintenance to detect amount of acne instead of using manual method lead to the developing of algorithm to use to the calculation for positions of acne on the patient's face and also detect and measuring amount of acne on the patient's face. The developing of algorithm is not accurate only on acne area, but it can also perform calculations amount of acne and comparative analysis of the difference between the recorded results in each time. It use the process and the basic steps from Image processing[1-3] to improve and apply. In this project was select the processes that involved significant of Face Detection[4], Blob Detection[5], and Color segmentation[6] to study the methods and procedures as well as the statistics to help determine the position, area, and color to increase the efficiency and accuracy of detecting. So the developing of algorithm to apply to the program and medical profession are taken from a variety of knowledge to generate new works to develop medical technologies. Technology that can apply to help in medical profession of facial skin in term of analyse the problem on each patient's face will be about the image processing and face detection. So mainly thing that necessary is picture of the patient that shown the problem of facial skin clearly. We observe from several research or previous studies about the Detection system, Edge detection, Face detection, and we know that the picture of patient's should be the same view point as possible to decrease the error that can happen when detect the key points (eyes, nose, and mouth) on patient's face. These technologies can find and detect some problem on the patient's face, for example, detect all spots, acne, wrinkles, etc. But it was a thoroughly method and difficult because of it maybe detect the unnecessary details around the face of patient s and also include the background of the picture that use to analyse. School of ICT, SIIT 1
detection. The original images and ground truth images experiment methods of blob Position mark image Ground truth image The images data in the real world position mark of the experiment method of wrinkles detection. Original image (1) Mark position image (1) School of ICT, SIIT 2
Original image (2) Mark position image (2) Original image (3) Mark position image (3) School of ICT, SIIT 3
Original image (4) Mark position image (4) Background In the globalization, improvement of technology, science, economy, society and the education provide people to have a better standard of living and lifestyles. Relating to the development of medical treatment facial skin that people become more attention of beauty and healthy, it affects to the rapid growth in beauty care industry, especially in now. Because of the people focus on the important of their own face would be good looking, clearly face and still younger, but the real natural face of people will be change over time. The natural acne always have a chance was born on the face. No matter how old you are or what gender you are. The medical of facial skin became important to treatment them. Since the majority of people are interested and enthusiastic to treat a skin disease. It is the acne on a facial skin that commonly found in the teenage age. The statistics from the Institute of Dermatology found that acne is one of the reasons to make a patient going to meet a doctor increasing. Therefore, the number of patients increased steadily with the number of doctors. The patients have several kind and different of people such as sex, age, body, face structure and the location of acne on a face. The doctor used a traditional tool to notes the result of treatment with written it by hand so the doctor will used the symbol of mathematic to draw a location of acne into the paper for represent an acne point instead of drawing into a real face such as draw a circle, rectangle or spot. The traditional way was found the problem when the doctor keep continue the treatment results in the next time, it doesn t work. The spot acne of patient is correct location and difficult to analyse the direction of quantity acne on the face. For each patient will spend a lot of time to treatment them so it effect to the management time that is difficult to manage more people in each day. The School of ICT, SIIT 4
doctors have only one way communicate with the patient by face to face. Instead of using the traditional way to retention the information of patient without drawing a mark point by hand. They would have a new innovation of medical facial skin program, to help a doctor and patient is easier and comfortable. For the unlimited technology always moved forward, it is the one key factor that effect in the process of human thought and analysis. The number of internet users and social media are expanded widely. Everybody can access to a large community that it cause to the human thinking about how to make themselves look good, because social network links to worldwide. Therefore, the most users always want to edit the images. To make face look smooth and clearly without acne on face. So the developer created and released a various application to respond them. The application was released many version to improve the result of images to be efficient. They blur the whole images to look smooth and unwanted point will disappear. This algorithm also made the environment around the face blur. All the cause of problem, the technology of computer satisfies all the aspect utility function of the doctor that can detect the quantity of acne. Therefore, we developed algorithm using some path of successful method to improve our program that the detection will have more accuracy. The research purposed, concerned to develop a medical facial skin for detect the acne on the face, and know which method has the efficient of detection. The important process and method involved to the Face detection, Blob detection and Color segmentation. Then, ours project used the knowledge of statistic in mathematic subject to adjust with the algorithm. The statistic method can be calculated the exact area and color that increase the accuracy of image processing. So that, the several major of knowledge will be applied to develop the tool of medical facial skin. We created an instance element of image to tests our method and evaluated the accuracy of each experiment to know the best solution. School of ICT, SIIT 5
Objectives The aim of this project is use of the image processing to develop software for read and analyse images of medical profession, especially the dermatologist. The doctor will be able to take the results of program to detect change and treatment acne on the face more effectively. The program reads the input images to detect only the problem area of acne or the different color area of surface, and analyse to correct the problem without unwanted area such as mouth, nose, eyes and hair. After detect it, will be able to analyse the amount of acne and calculate for represent in the percentage in order to compare the different result with the same images. The doctor will know how the movement of treatment going to be and can analyse the next treatment results. This program useful to the doctors, they don t need to meet their patients. The programs used only the image of face patient's then detect and display the specific acne area. So, doctors don t need to analyse the problems by themselves, they will know immediately how to cure their patient's problems. Moreover, the technology of image processing further to developed the mobile application that related to beautify the images such as photo editor, photo blur and beauty face photo etc. The algorithm adjusted to the one part of application to be increase efficiency that is changing different tone of the surface and focus only the specific point. Outputs and Expected Benefits 4.1 Outputs The output that we want is ground truth image of patient's face from detected process. The image will show only the line of wrinkles on the patient's face correctly. So, the important thing that we concern is how to write a program that can read and detect only the wrinkles on the patient's face because the images of patients is composition of face, hair, background, etc. So, we want to make sure that program will not detect something else that not a wrinkle. We learn from the previous researches that maybe program will detect some key point on the human face (eyes, nose, and mouth) and hairs that have a line shape similar to the wrinkles The direction of human face also can effect to the program that it limited for program to analyse the image. Every images that import to the program, the human face have to be in the same direction and the best is the human face that look straight to the camera because it easier to the program to read and detect the wrinkles. 4.2 Benefits Our project, we write a useful program for the medical profession, especially in facial skin line. The program will useful with doctors to analyse the methods to cure their patient's problem by using the ground truth image from program that show the lines of wrinkles on patient's face. This can decrease the problem that nowadays we lack of doctors and they not have a chance to discuss or analyse the patient's problems. Doctors can use the ground truth image from program to follow up the change of the patient's problem in every period of times, they can know that the problems is better or not by compare the ground truth images to see the changes of the wrinkles. School of ICT, SIIT 6
The future development of our method can apply with the smart phone on the application photo feature. The program will detect the wrinkles on human face then it can develop to blend/blurred the wrinkles that program detected to make the images more beautiful. Literature Review Several ideas and methods that we have study would be the guideline for our project of acne detection. Image processing [1] Automatic Facial Skin Defect Detection System : processes of the proposed approach, which includes face detection, facial feature detection, ROI locating, spot detection and wrinkle detection. Image processing [2] Extraction of acne lesion in acne patients from Multispectral Images : the algorithm for the extraction of acne lesions from MSI. In the preprocessing, background, hair and normal skin are removed while in the classification step, reddish papule, pustule and scar are classified. Image processing [3] Learning-Based Detection of Acne-like Regions Using Time-Lapse Features : Detecting Acne-like Regions In Skin Images that use algorithm to detect acne lesions using images acquired under cross-polarized modality. Face Detection [4] The face detection separates into 4 methods; Skin color segmentation is the process of rejecting non-skin color from the entire image. It is based on the color of all races human face, Morphological Processing is to performed the clean up of the image. The goal is to end up with a mask image that can be applied to the input image to yield skin color regions without noise and clutter, Connected Region Analysis the output from morphological processing still contains non-face regions. Most of them are hands, arms, legs, clothing that match the skin color and some parts on background. In connected region analysis, image statistics from the training set are used to classify each connected region in the image. Template Matching is the basic idea of template matching is to compare the image with another template image that is representative of faces. Finding an appropriate template is a challenge since ideally the template (or group of templates) should match any given face with differences of the size and features. Blob Detection [5] Automatic detection of blobs from image datasets is an important step in analysis of a large-scale of scientific data. These blobs may represent organization of nuclei in a cultured colony, homogeneous regions in geophysical data, tumor locations in MRI or CT data, etc. Color Segmentation [6] It is based on the color of all races human face. We set the HSV (Hue, Saturation, Value) color space for segmentation up for only focus on that specific value of H and S. This information was used to define appropriate thresholds for H and S space that correspond to faces. The threshold values were embedded into the color segmentation routine. School of ICT, SIIT 7
The skin color segmentation resulting image is converted to HSV color space. All the color pixels that fall outside the H and S range are rejected as the non-face objects. Methodology 6.1 Approach To developing algorithms of detect facial acne, we started to collecting a face acne images of patient. Those images used to simulate the detection point acne in the Photoshop program. We imported the image to first layer, and created the second layer to spot a specific area of acne. Then exported only the paint of spot acne to be a result of the simulation (Ground truth) that obtained in this process, using it compared with the results of the testing algorithm in computer processing. Ground truth represents ability of the human detecting. School of ICT, SIIT 8
To processing the algorithm, import the images into the program and it will automatic detection the acne. 1. Convert the RGB color images to the Grey scale images 2. Find the maximum value of intensity images with X and Y coordinates on the Grey scale images. 3. Calculate normalized grey-scale image by divide the value of intensity to 0 or 1 with X and Y coordinates, to compare with HSV images. 4. Retrieve HSV color images to define the value of H(Hue) = 0 for drop a red color 5. To extract the brightness area (V) from HSV model and define Dark color = 0 and White color = 1. 6. To subtract by V-Grey scale, the result show the region of maximum lightness 7. Define the value of threshold background is white color otherwise will be a black color. The images convert to negative binary color 8. To analyse the images for eliminate a tiny spot area. 9. From the result of step 8, divided the area less than 7000. 10. The results from step 7, 8 and 9 will represent the appropriated size of specific object. 11. Create the square to cover the area of detection 12. Detect the input images and calculate the amount of acne. School of ICT, SIIT 9
The processes of the program consist of two main techniques that include the Face detection and the Statistic method. The defaults method is to upload the image of face patients into the system. Then the program will check the acne on the face through two methods that describe in the previous statement. The both image results are different accuracy away. Comparing it with the evaluation processed (Evaluation Accuracy) by Qualitative and Quantitative data. The structure working process of the program consists of two main techniques. There are Statistic Methods and Face Detection by way of starting are the same. But the results in term of accuracy will be difference by can compare the image from the results of the program (Evaluation Accuracy). As the result image shows that the amount of acne on the patient's face from the treatment of each time. Using basic manually method to checking each frame of acne that program detected is correct or not. By the way called Qualitative and Quantitative and each method are different, as follows. -Qualitative: Evaluation of Qualitative data is considered on the details of the images, the result is information that can be seen with the naked eye. Evaluation from the result images by comparing each image and determine of each frame of acne that program detected is the actual acne including areas where error of program detection. -Quantitative: Evaluation of Qualitative data is a consideration about numbers or data that can be measured as the number of digits. From each image results can be evaluated by calculating the quantity of acne on the patient's face and determine whether the average of the frames of acne that program detected is the actual acne including areas where error of program detection. School of ICT, SIIT 10
From the analysis of Quantitative and Quantitative, each result images will have different accuracy. The result of analysis will be classified into 4 types (Confusion Matrix)[7]; - True Positive (TP) is what the program predicted, and says it is true. - True Negative (TN) is what the program to predict that's not true, and says it is false. - False Positive (FP) is what the program predicted, but says it is false. - False Negative (FN) is what the program predicted that's not true, but says it is true. From the results, each type of classification that is used to calculate the "true positive rate" called Sensitivity and Specificity[8] which calculate the rate of accuracy of the results from the program. Sensitivity is the rate of what the program predicted, and says it is true, also can be calculated as a percentage. And Specificity is the rate of what programs predict that's not true and says that it is false, also can be calculated as a percentage. School of ICT, SIIT 11
To summary, the structure working process of the program consists of the two main techniques, Statistic Methods and Face Detection by way of starting are the same that to upload images of acne on patient's face into the system, but difference in term of accuracy of the results by can compare from the evaluation precision(evaluation Accuracy). We will comparing from the result images from program. The results of the program can tell changes or the development of acne on the patient's face by a method called Qualitative and Quantitative. Each method will be able to evaluate the results come out differently. From the results of each type of classification can be divided into Sensitivity and Specificity those are the calculation precision of the results of the program. Sensitivity is the rate of what programs predict that true and said it was true. Specificity is the rate of what programs predict that false and said that it's not true, both can calculate out as a percentage. 6.2 Tools and Techniques 6.2.1 Tools Software - Adobe Photoshop : To simulate the expected result of acne images for compared with the results of experiment of program - Matlab : To write a program with a basic function of image processing command Hardware - Camera 1 unit - Computer 1 unit Functional Specification - Upload image of face acne - Represent to detected area of acne - Represent a number measurement of acne 6.2.2 Techniques The technique using to find the solution of the method is less a mistake, but most efficiency and more accuracy is following to. 1. Statistic Methods The detection using image processing technology, in each figure of images represents the shapes of square surrounding the specific point of acne that is a blob. The blob determine the scope area of acne problem that is the way to used it developed program have more accuracy. This method calculated the average area and color of acne problem after that classify the different of data into the group. Then eliminate the group that is doesn t close with the other group. Those areas are the mistake of detection program, and assume it is unwanted acne area. 2. Face Detection The technology of face detection always continues development. The experiences of program have more accuracy. The basic method is detected the region of face structure such as eyes nose or mouth to locate the right position that you need. Moreover, the researches show the Connected Component Analysis method to solve the detection mistake. School of ICT, SIIT 12
In each object on the body, define the identical value of each path of body then set the value of data that we want to deduct it. So program doesn t detect the identical area and represent only the region of face. Those methods can reduce the detection mistake and can modify it to use with only the face structure to less mistake of acne detection. School of ICT, SIIT 13
Project Schedule Our work schedule is based on the advice of Dr. Bunyarit Uyyanonvara(advisor) in every week, as well as the duration of the study the research and also development of program. Task Description Person Duration Deadline Status 1 Review the previous study and research. Select which one can be the guideline for our project 2 Try to learn and study Matlab program 3 - Design the progress. -Select the methods to implement algorithms of project then try to work on Matlab 4 Prepare report and paper for the conferences (NSC & IC-ICTES 2015) 5 -Develop program to split the blob of acne. (1blob per 1acne only) -Develop program to count and show amount of detected acne 7 Prepare slides for midterm presentation 8 Submit the report to NSC conference 9 Learn the progress of the results images from program 10 Prepare slides for final presentation 11 Submit paper to IC-ICTES 2015 conference 12 Submit proposal and slides for final presentation 1w 1w 2w 1w 2w 1w 2w 1w 10d 2w 1w 18 Sep 14 25 Sep 14 100% completed 9 Oct 14 100% completed 16 Oct 14 50% completed 30 Oct 14 100% completed for count and show amount of detected acne 6 Nov 14 100% completed 10 Nov 14 17 Nov 14 27 Nov 14 100% completed 80% completed 50% completed 8 Dec 14 100% completed 15 Dec 14 100% completed School of ICT, SIIT 14
References [1] Chuan-Yu, Shang- Cheng Li, Pau Choo Chung, Jui- Yi Kuo, Yung- Chin Tu. "Atomatic Facial Skin Defect Detection System." Dept. of Computer Science & Information Engineering, National Yunlin University of Science & Technology, Taiwan. pp.527-532, 2010. [2] Hideaki Fujii, Takashi Yanagisawa, Masanori Mitsui, Yuri Murakami, Masahiro Yamaguchi, Nagaaki Ohyama, Tokiya Abe, Ikumi Yokoi, Yoshie Matsuoka, and Yasuo Kubota. "Extraction of acne lesion in acne patients from Multispectral Images". Annual International IEEE EMBS Conference Vancouver, British Columbia, Canada. pp.4078-4081, 2008. [3] Siddharth K, Madan and Kristin J, Dana. "Learning-Based Detection of Acne-like Regions Using Time-Lapse Features". Department of Electrical and Computer Engineering, Rutgers University NJ, USA. 2011. [4] Phannapat S, Watcharaporn S, Maetawee J, Guntachai O. "Face Detection" School Of Information Computer and Communication Technology Sirindhorn International Institute of Technology, Thammasat University, Thailand. 2013. [5] Anne Kaspers. "Blob detection". Biomedical Image Sciences,Image Sciences Institute, UMC Utrecht, 2011. [6] Chai D, Ngan K.N., "Face segmentation using skin-color map in videophone applications," Circuits and Systems for Video Technology, IEEE Transactions on, vol.9, no.4, pp.551,564, Jun 1999 School of ICT, SIIT 15