Lisbon Pole. Torre Norte, Piso 7 IST, Av. Rovisco Pais Lisboa Portugal

Transcription

1 INSTITUTE FOR SYSTEMS AND ROBOTICS Annual Report 2001 Lisbon Pole Torre Norte, Piso 7 IST, Av. Rovisco Pais Lisboa Portugal Tel Web Server: Fax

2 Director Prof. João José dos Santos Sentieiro Vice - Director Prof. Maria Isabel Lobato de Faria Ribeiro Torre Norte, 7º Piso IST, Av. Rovisco Pais Lisboa Portugal Tel.: Fax.: jjss@isr.ist.utl.pt

3 Table of Contents OVERVIEW 1 1. INTRODUCTION Reports on visit to ISR-Lisbon of the Advisory Committee Report by Prof. Henrik Christensen Report by Prof. Panos Antsaklis Report by Prof. Mos Kaven RESEARCH TEAM AND INTERESTS Members and Collaborators Current Research Interests Intelligent Control Computer and Robot Vision Artificial Intelligence and Manufacturing Systems Mobile Robotics Signal Processing Evolutionary Systems and Biomedical Engineering Dynamical Systems and Ocean Robotics Aeronautics RESEARCH ACTIVITIES Research Projects Robotics Underwater and Ocean Robotics Cooperative Robotics Land Mobile Robotics Robotic Manipulation Image Processing Underwater Acoustics Computer Vision 47 i

4 D Environment Reconstruction Artificial Intelligence Biomedical Engineering Operations Management Data Networks Communications Aeronautics Post-Docs Activities Report Activity Report of Didik Soetanto Activity Report of Lionel Lapierre Activity Report of Sergey Rumyantzev Activity Report of Sandra Clara Gadanho Theses Theses Concluded During Theses in Progress During Advanced Training Courses Seminars Reading Groups Visits Abroad Supervision of StudentsEnrolled in Foreign Universities Congress, Meetings and Presentations Invited Talks Participations Service Activities Editorial Boards Advisory Boards Programme and Technical Committees Chairperson Reviewers Other Activities 96 ii

5 3.7 Academic Activities Distinguished Visitors Special Events NARVAL Sea Trial INTIFANTE MONAZ Sea Trial Awards Publications Submited Pappers LABORATORY FACILITIES AND SERVICES Common Facilities Laboratory Facilities 121 iii

6 iv

7 OVERVIEW Since its foundation, ISR (Lisbon) has given special attention to international cooperation in order to strengthen and broaden its scientific competence. Two types of cooperation are especially noteworthy: firstly, participation in R&D projects in conjunction with universities, research centers, and European businesses of note under the auspices of programs funded by the Portuguese Science and Technology Foundation, European Community and other agencies; and, secondly, training initiatives, primarily through master and doctoral programs. These initiatives have involved not only the Instituto Superior Técnico (IST) and the University of Algarve (UA) but also universities and research centers in Europe and the United States. In 2001, ISR-Lisbon in partnership with other 3 research units (o Centro de Estudos em Inovaçâo, Tecnologia e Políticas de Desenvolvimento do Instituto Superior Técnico (IN+), o Centro de Recursos Minerais, Mineralogia e Cristalografia (CREMINER) da Universidade de Lisboa e o Centro do IMAR (Instituto do Mar) da Universidade dos Açores) has submitted to the MCT a proposal seeking to be considered an Associate Laboratory, a new status only awarded to research units which have been evaluated as Very Good or Excellent by an international panel of evaluation and whose objectives are considered by the MCT to be of national interest. The proposal was approved and ISR-Lisbon was awarded by the Ministry of Science and Technology the status of Associated Laboratory and has seen his budget to be increased in order to accommodate the possibility of contracting 14 new Ph.D. researchers during the next five years. In this new context the ISR Lisbon (Associated Laboratory) strategic objectives have been revisited and focused in terms of the following research and development areas: A Technologies for Ocean Exploration B Monitoring and Surveillance using Robotic Agents C Sustainable Technologies and Environment Systems D Signal Processing for Communications and Multimedia Networks In order to push theoretical developments in the fields of marine robotics, underwater acoustics and communications, Computer Vision and Cooperative Robotics efforts will be made to promote international operation through joint projects; to reinforce the teams with full time and post-doc researchers; to bridge the gap between theory and practice by pushing the collaboration with marine scientists, environmental experts and government agencies interested in the management of ocean resources and civil protection; to endow researchers with the infra-structures and technical personnel required for the development and testing of ocean equipment and vehicles. In 2001 the Systems and Control Theory Group has launched a number of activities including seminars and two advanced courses on Optimization Theory and Dynamic Stochastic Estimation, Prediction and Smoothing. The seminars and courses were attended by a large number of Ph.D. students and faculty. The objectives of the Systems and Control Theory Group are: (i) to make up for the lack of advanced graduate courses in control theory offered by the University; (ii) to attract young students with a keen interest in theoretical developments in systems and control; (iii) to provide a forum for cross fertilization of ideas in emerging theoretical areas and; (iv) to give support to practical developments in challenging areas of systems' engineering. The strengthening of the other ISR well established areas of research will be obtained by a better dissemination of research results within the international community; an increased participation in international scientific networks; the reinforcement of collaborative links with industry; and pursuing an aggressive policy of attracting foreign post-doc and Ph.D. students. In 2001, 141 senior and junior researchers have developed their research activities within ISR (Lisbon Pole). These included 20 professors, 1 Principal Researcher, 4 post-docs, 51 Ph. D. Students, 26 M.Sc. Students, 35 undergraduate research trainees, and 4 Research engineers. Of the 51 Ph.D. Students, 19 are Teaching Assistants at the University or at other Higher Education institutions and the remaining 32 are grant holders (17 Portuguese and 15 foreigners). Of the 26 M. Sc. Students, 4 are Teaching Assistants at Higher Education institutions. 1

8 During 2001, the institute researchers have been involved in a large number of national and international R&D projects, financial resources being provided at a national (FCT, ICCTI, Ciência Viva, private companies) and international level (EU, and others), contributing to increase the international visibility of the institution. As a result of these activities 11 M.Sc. and 3 Ph.D. theses were concluded, 5 papers where published in books, 22 papers were published in well known international journals, and 78 papers presented at prestigious international conferences. Steps will be taken to encourage researchers to increase the publication of their research results in archive journals. Also the number of publications by Post-Docs is, in general, beyond expected and hence efforts should be made by their supervisors in order to change the situation. The training of young researchers has pursued, involving 54 Ph.D. Students (Portuguese and foreigners) and 37 M. Sc. Students (Portuguese). Three researchers have concluded their doctoral theses, and 11 researchers have concluded their M. Sc. theses. Also, young licentiates from several European countries have come to participate in short and medium term research initiatives. Several researchers have stayed short or long periods abroad, as visiting or invited professors, researchers or students. The participation in editorial boards of international journals, and in the program committees of international conferences of high reputation was also very active. Several foreign senior and junior researchers have visited and stayed with ISR. Internal efforts were made to increase the cooperation links between different groups, in particular institute wide seminars were organized on a weekly basis with required participation of the students. Also and following comments made by the international evaluation panel the AIMS Lab and the IC Lab have started a discussion process aiming the merging of the two groups in a single Lab to be denominated Intelligent Systems Lab. ISR is also organizing the 10th Mediterranean Conference on Control and Automation (MED2002) that will take place at IST in July 2002 with the support of the IEEE Control Systems Society. As closing remarks we would like to stress the fact that the international visibility of ISR and in particular of some of his groups has increased significantly. This fact translates not only on the increasing number of foreign graduate students and post-docs wishing to pursue their research carriers at ISR, but also on the international contacts leading to collaboration projects and in the interest that Workshops and Conferences organized by ISR are raising in the scientific community. Finally we urge again the host institution, IST, to recognize the research and graduate advising contributions of the faculty through differential classroom teaching assignments and trough the increase of administrative and technical support for all our laboratories. João Sentieiro ISR, April 30,

9 1. INTRODUCTION This report describes the activities developed, during 2001, by the ISR (Lisbon) members and collaborators. In terms of background the ISR (Lisbon) researchers come from two main areas: Electrical and Computer Engineering, and Mechanical Engineering. This report is organized in four main chapters: The introduction chapter includes the reports of the Advisory Committee members. In chapter 2 the research team is identified and the research interests stated. Chapter 3 contains a full report of the activities of the members affiliated with the Department of Electrical and Computer Engineering of Instituto Superior Técnico (IST), with the Exact and Human Sciences Unit of the University of Algarve (UA) and with the Department of Mechanical Engineering of IST. This chapter is organized in subsections covering all relevant aspects of the research and academic activities pursued by ISR (Lisbon) researchers during 2001, namely research projects, supervision of Ph.D.and M.Sc. theses, courses and seminars, visits abroad, invited talks, editorial boards, program and technical committees, journal and conference reviewing and publications. In chapter 4 a description of the laboratories equipment and other facilities is provided. 1.1 Reports on visit to ISR-Lisbon of the Advisory Committee Due to agenda difficulties and to the 11 th of September events it was not possible, this time, to have all the members of the Advisory Committee (AC) visiting together the institution. Three visits were organized : two in October 2001 (Prof. Henrik Christensen and Prof. Panos Antsaklis) and one in February 2002 (Prof. Mos Kaveh). The three reports produced by the Committee members are presented in the following sections Report by Prof. Henrik Christensen GENERAL IMPRESSIONS RESEARCH QUALITY Much of the research is of international quality. A few examples include excellent research of significant international interest. A few efforts might not be of international quality and it might be important to focus research to ensure adequate resources are available. RESEARCH FOCUS More explicit formulation of research focus/issues for each laboratory is desirable. The overall outlines are very useful but they are fairly generic. (The description by the DSOR lab is a good example of a more detailed statement, but concrete shorter term results/focus would be useful) Diverse, yet, strong research on most aspects of robotics. RESEARCH MANEGEMENT Visionary and open management structure that encourages the pursuit of strong research agendas that are theoretically well-founded. Strong support for the different groups. The management seems to provide a most useful infrastructure for the largely independent research within each group. Overall management of the institution is very good. Given teaching requirements it is impressive to see an institution that has managed to strengthen its research effort to a level of good international recognition. OBSERVATIONS International students might not be as closely integrated into the institution as one might wish for. Foreign placement for tenure track personnel. To ensure that tenure track personnel have a solid international perspective it is considered important that all junior researchers prior to entering the tenure track programme have spent a period of more than 6-12 month at a foreign research institution. 3

10 Engineering resources needed. Significant resources by researchers and students seem to be used for basic engineering. A certain amount is desirable, but for extensive engineering it might not be efficient to use research time. Ambient intelligence should be considered. The focus of the next framework programme is "Ambient Intelligence". The diverse research platform at ISR might make it an important candidate for participation. In addition it might be important to monitor the possibility of participation in a "network of centers of excellence" The theory group has been setup. The group provides seminars etc. It is considered essential to strengthen the group and ensure that it is efficiently interfaced to the other research groups. The heavy teaching duty of faculty is still a major obstacle to widespread excellence. Only through unreasonable work efforts is it possible to maintain/setup an excellent research environment. 1. MOBILE ROBOTICS The mobile robotics group is lead by Assoc Prof Maria Isabel Ribeiro and has a new assistant professor Joao Sequeira. In addition the group has 2 PhD students, and 2 MSc students. The research is focussed on mobile robotics. At present there are two major avenues of research: outdoor robotics and cooperative robotics. In outdoor robotics the emphasis is on mapping using topographical models as a basis. The topographical models are augmented with local feature models. The focus is at present on basic mapping. The research is carried out in the context of a new outdoor vehicle (RWI ARTV-jr). Implicitly the problem addressed is the fundamental problem of simultaneous localization and mapping, which traditionally has been formulated as a non-linear Kalman filtering problem. Through reformulation of the problem it is possible to use local models; this does however require development of local mapping techniques and methods for robust place recognition. The use of a variety of sensory modalities is required to address this problem. The project thus presents an excellent platform for studies of multi-sensory fusion. The use of GPS opens for studies of use of sensors that have stochastic availability, and non-deterministic delays. The platform can also be used for studies of fusion vision, inertial sensors and laser ranging. Overall a number of interesting results are expected in this area. One concern might be focus. To enable progress and pursuit of world class results it is recommended that a strong focus is defined for the project. General outdoor robotics is today a highly diverse area and it is considered difficult for a relatively small group to address many of these issues, consequently it is recommended that a specific focus is adopted. The other major problem addressed within the group is cooperative robotics. Here in particular simulation a study of loose coordination of teams of robots has been studied as a basis for definition of strategies for collaboration. The problem of robot cooperation is interesting as it can encompass a number of diverse issues in terms of distributed sensing, local communication, ad hoc utilization of resources etc. Cooperation has frequently been addressed in terms of local interaction using basic control models such as impedance control. The idea behind this project is to project sensory derived model into a control manifold that can ensure coordinated motion of a team of robots. This is an interesting approach to coordination, but it might be useful to compare it to traditional methods so as to demonstrate it superiority. Again the problem of cooperative robotics is highly diverse and encompasses a large number of issues that only can be addressed in a meaningful manner through a strong focus on a particular sub-field. Once such area might be distributed sensing, communication models for coordination, ad-hoc resource management, etc. In addition to the two major projects the group also has a number of exploratory projects on robotics and IT, and robots for inspection. Both projects are relatively small and meant to explore potential avenues for future research. The issue of IT systems and their use of robotics as an integral component is most timely. This is at present an issue of significant industrial attention. The integration of complex systems is by no means a trivial problem. There are however a number of existing standard for construction of systems and in addition it is necessary to consider all the diverse aspects of IT systems for automation to make real progress. The issue of robotics for inspection is another interesting problem that potentially might be of industrial interest. This is however a highly competitive market and it might be difficult to compete with large groups and industrial companies. None the less this is a good topic for student projects. The group has a good collaboration with the intelligent control laboratory. Overall the group seems to be on its way to change focus from laser range based modeling to the areas of outdoor navigation and cooperative systems. With proper focus on key issues the group has a potential to build up a strong competence in these areas. It might be useful to consider the set of international links with key research groups in USA (CMU, MIT) and Europe (Örebro, EPFL, Karlsruhe, CLAWAR). 4

11 2. DYNAMIC SYSTEMS AND OCEAN ROBOTICS GROUP This group is headed by Associate Prof Antonio Pascoal. In addition Assistant Prof. Carlos Silvestre and Assistant teacher Paulo Oliveira, play senior roles in the coordination of the research. The group has 3 post doctoral researchers, 4 PhD students, and a large number of MS. c students. The focus of the group is on dynamic systems and its application to oceanographic robot vehicles. The research is founded in well founded linear and non-linear models of vehicles and their control. For control of these vehicles hybrid control models encompassing discrete events (modeled using Petri-nets) and linear control systems have been studied. The work has involved the design of a surface vessel (DELFIM), an oceanographic vehicle (CARAVELA) and a submersible (INFANTE). An interesting part of this research is the holistic approach to the research. The effort involves all aspects from theoretical modeling over algorithmic studies to engineering of operational systems. At the same time the systems include sensing, actuation, communication, electronic and real-time software development. This is a truly impressive effort. It is judged that the group is making solid progress on both theoretical issues and their empirical evaluation. The system has been tested in number of different settings. Recently the group has also undertaken work on control of helicopters. The group has so far developed basic dynamic models for flight and is now ready for studies of sensor driven mapping and navigation. It is not immediately obvious why the group has chosen to expand their domain of application to aerial vehicles. In addition to theoretical studies and engineering of systems the group is also actively involved in experimental use of the platforms for mapping and exploration with geologist and mine biologists. These studies have in particular been carried out at the Azores. The group seems to have a particularly strong network of national and international collaborators. It is, however, not immediately obvious that there are good collaborative links to other groups within the institute. As an example both this group and the computer vision group have projects in underwater navigation, yet it is not obvious that complementary expertise is utilized. There is little doubt that this is a very strong underwater robotics group that likely is among the best in Europe. One concern is that the group is spending a significant amount of its resources on engineering in terms of electronic design, implementation, basic software design (home made software at all levels), and systems design. Funding of such activities might be difficult to obtain and it might be possible to sub-contract such engineering work to outside companies. It is encouraging to see the solid theoretical basis and its implementation into operational systems. The group also seems to be reasonably well funded. 3. COMPUTER VISION LABORATORY The group is headed by Prof. Joao Sentieiro, with strong support from Assistant Prof. José Santos-Victor and Assistant Prof. João Paulo Costeira. The group involves about 10 PhD students and 2-3 MSc students. In addition the laboratory has several visiting students. The group does research on basic computer vision methods in particular based on motion and binocular cues. The information is used for generation of world models, and for navigation in the context of aerial, underwater and ground vehicles. More recently the research has also involved biologically inspired methods for world modeling and control. The research involves a strong combination of signal processing, geometry and control. The research can be divided into four main directions: i) active vision, ii) vision for navigation iii) 3D reconstruction and iv) biologically motivated methods in vision. 5

12 The active vision work is directed at use of space variant sensing for ocular motor control. The research involves a combination of cue estimation, tracking and motor control. This is a long-term research effort within the group and the researchers involved have a strong record in this area. At present the research is aimed at computation of cues for front-end like representations as seen on human (V1-V5). The work involves disparity channels and band pass filters for motion estimation. In addition various segmentation methods are considered. Overall the work is interesting and of good quality. It would however be interesting to see a systems perspective on the work carried out. i.e. it is not immediately obvious that issues such as complexity, task dependency etc. are taken into account for this work. In addition it would be of interest to see integration with higher level processes where the issue of memory and representations becomes more pronounced. The work on vision based navigation is one of the long-standing research challenges addressed within the group. A majority of the research is at present directed at use of omni-directional camera systems for navigation. A rich variety of topics are studied here including: perception-action integration, world representation, sensor design, learning of motion models and navigation strategies. It seems that a significant part of the work is directed at use of sensors that are composed from specially designed spherical mirrors in combination with space-variant sensors. Through this combination it is possible to design sensors according to task constraints. This is a more interesting area of research. It would however be of interest to see a more explicit formulation of task dependencies. As to the navigation the work is based on construction of a mixture of iconic and topological maps. The work is here a combination of semi-automatic mapping techniques and methods from pattern recognition. Overall the work is interesting and of solid quality. It would be of interest to see evaluation of these techniques in comparison to well established methods for in-door navigation. It is not obvious that thorough evaluations and metrics has been defined for this research and it is consequently difficult to assess the progress and/or benefits with respect to state of the art. In terms of 3D recovery of models of the world based on monocular images and or binocular images two major lines of research have been pursued. The first line of research involves use of geometric constraints in combination with semi-automatic recognition of features for recovery of geometric shapes for polyhedral objects. The work utilizes constraints such as symmetry, co-planarity, planarity, linearity. This work is considered of high quality and largely in accordance with state of the art. The other effort within this area is related to optimal matching of pairs of images in the presence of outliers. The work is based on feature matching across pairs of images. The strategy is to consider a structured approach to selection of permutations of matching pairs to ensure a structured (non exhaustive) traversal of the search space. The work appears to be highly interesting but it would be of interest to see a comparison of this approach to some of the many methods (both optimal and sub-optimal) published in the literature. This is by no stretch a new problem. The fourth area is related to biologically motivated methods for perception-action integration in particular related to learning by imitation and distributed motor representations. This is a natural extension of the research mentioned above and the major direction is here towards learning of presentations and control strategies. Both areas are of significant interest and most timely. The research is still at its infancy and it is too early to determine the potential of this research. Overall the research is of good quality. One concern is that the group is undertaking a large number of research issues, and it is not immediately obvious that the different efforts have a common theme. It will be difficult to achieve international quality in the research without some focus. The problems could be solved through involvement of more academic staff or post-doctoral researchers. Alternatively the research could be focussed on a smaller number of topics. The group has a strong set of international collaborators, which is highly encouraging. At the same time there is some collaboration with other groups within ISR, but would be of interest to see more work with groups such as the Dynamic Systems and Ocean Robotics Group. 6

13 1.1.2 Report by Prof. Panos Antsaklis I visited ISR on Wednesday and Thursday, October 17 and 18, 2001 and I met with two research groups on Wednesday (Intelligent Control, AI and Manufacturing Systems) and with three on Thursday (Evolutionary Systems and Biomedical Engineering, Dynamical Systems and Ocean Robotics, Aeronautics). On Thursday I also had a one hour long meeting with a group of Ph.D. students and Post-docs as well as a one hour meeting with Senior Researchers from the groups I visited. In addition I met with Carlos Bispo and had several meetings with Joao Sentieiro and Michael Athans. The contents of this report are as follows: The first part of the report summarizes the overall comments and suggestions regarding the research quality and focus; it also contains general comments and suggestions from the interviews I conducted with Graduate Students, Post-docs and Senior Researchers. The second part of the report contains brief comments and recommendations regarding individual research groups. PART I GENERAL COMMENTS AND RECOMMENDATIONS Below please find a summary of my observations and recommendations regarding the five research groups I visited. The range and quality of research is significant, mostly of international caliber and in several cases truly impressive. The senior researchers as a group are very able and competent. Good work is being done and the ISR researchers should be proud of their accomplishments. The research activities involve a healthy mix of theory and practice-with some exceptions. This I think is the great strength of ISR and it distinguishes it from other university research institutions in Europe and internationally. It is apparent that the overall research leadership is excellent. The environment is friendly and amenable to research and scholarly activities. The new theory group is up and running. Current courses offered are in optimization and estimation. Novel research efforts include possible projects in mathematical biology and bioinformatics with emphasis on the mathematical modeling of the immune system. Such efforts should definitely be encouraged with additional resources. I hope that more people from ISR will be involved in them. Communications between laboratories could be improved. Some groups are interacting with the others, other groups do not interact as much. The Aeronautics group does not interact with anybody else in ISR. My recommendation is either to drop the Aeronautics group from ISR or better perhaps, to bring in a second faculty member (senior researcher) whose aim will be to establish within one year common ground and submit a joint proposal with other ISR groups. There is urgent need to improve infrastructure particularly Internet access speed, computer network administration and technical support, hardware and especially software support. I would recommend that ISR secures the services of a computer software professional who would be able to help the research groups and provide network support for ISR. It is highly recommended that a regular distinguished visitor lecture series be established. Perhaps a name could be attached to the series and outside funding may be secured. To improve opportunities for interaction, have regular seminars and regular coffee/tea hour. Use to notify everyone about seminars and post notices on website. It may be a good idea to post resources and events (books, journals, presentations, tutorials) of each lab on the website. 7

14 The increased support from Portuguese ministry of science and technology is good but not enough. Emphasize European projects primarily to gain visibility. Establish mechanisms to provide researchers relief from the heavy administrative burden of EU projects. A way to improve visibility is to organize international workshops. One such effort is currently under way (MED'02 Conference in Control and Automation) It appears that in each group the number of projects, the variety of project topics, the fact that many projects have a time-consuming hardware component, the number of graduate students and the significant teaching obligations do not leave enough time for the group leaders to identify and explore new promising research directions, to go after new large European research projects, and to build research teams for European level proposals (such activities not only bring in significant resources but also provide international visibility and a forum for the work at ISR to be measured against related work around the world). Ways to correct this serious drawback include reducing teaching loads, increasing the number of faculty in each group, focusing on fewer bigger research projects and providing infrastructure support in the form of software computer expertise. FEEDBACK FROM THE GRADUATE STUDENTS AND POST-DOCS On Thursday October 18, I had a one hour meeting with Ph.D. students and Post-Docs from the 5 labs I visited. First, the students were asked to list on a sheet of paper 5 good things about ISR, 5 areas that they felt needed some improvement, and where they plan to be in 3-5 years. The following is a summary of the 16 responses: Positive aspects of ISR: 1) The students recognized the expertise of the advisors and faculty, considering this a strong asset. 2) The students praised the reading groups and the opportunity to work with other students. 3) The variety of projects and research opportunities was another positive aspect because the different interests of the students are fulfilled. 4) Some of the students were pleased with the availability of resources, including the libraries and online journals. Aspects of ISR that could use improvement: 1) While some of the students praised working with an advisor, others mentioned that they wanted more personal attention from their advisors. This might just be a problem with one or two of the advisors, and not a complaint across the board. 2) Many students complained about internet speed. 3) While the students enjoy working in groups, some suggested that there be more social interaction among people, as well as interaction with other teams. They also mentioned simply wanting to know what the other teams were working on - perhaps there could be more communication among the groups. 4) Some of the students complained about the lack of available (library, books, journals) resources, as opposed to the group above who were pleased with the options. It was also suggested that language classes (Portuguese) be offered to new foreign students. 5) Some students expressed a desire for more opportunities to speak internationally, as well as having more international speakers on a regular basis. 8

15 Then I invited them to bring up their comments, concerns and suggestions. Certainly there is significant overlap between the written and oral responses, as expected. Here are some of their comments. 1. Organize Portuguese language courses for new foreign students. This is very important if you want to have foreign students. 2. We need more courses offered by ISR for example on nonlinear control. Right now there are only courses on optimal control and estimation. 3. Some labs have reading groups. This is a good idea to be adopted by all groups. 4. Have regular tea-time to socialize and meet everyone. 5. Have regular seminars. It would be great to have distinguished lecturers on a regular basis. However, Professors need to show up at the seminars. 6. There is lack of integration in ISR. Some groups do no talk to any other ones. 7. Participation in international conferences by students should be encouraged. 8. The internet facilities are very slow and need to be improved. 9. Need cooperation with other universities in Portugal. FEEDBACK FROM SENIOR RESEARCHERS On Thursday October 18, I had a one hour meeting with Senior Researchers from the 5 labs I visited. Here are some of their comments: Improvements in infrastructure are needed. Most important is the upgrade of internet access. It appears that the access speed to the internet can sometimes be painfully slow leading to delays and frustration and to significant amounts of work loss. It appears that air conditioning and sometimes heating are effectively non existent. This situation also leads to work loss. Hardware and primarily software infrastructure support is needed urgently. Identity of ISR involves a healthy balance between theory and practice. This should be continued and encouraged further. There were concerns regarding library facilities-some groups have them other groups have no access. It was felt that IEEE digital access is important to achieving their research goals. As some groups have grown it is important to plan for new physical space for laboratories and offices. PART II The second part of the report contains brief comments regarding individual research groups. I met with two research groups on Wednesday (Intelligent Control, AI and Manufacturing Systems) and with three on Thursday (Evolutionary Systems and Biomedical Engineering, Dynamical Systems and Ocean Robotics, Aeronautics). Intelligent Control Laboratory Leader: Pedro Lima Assistant Professor Research topics of interest to the Intelligent Control group include hybrid systems, multi-robot systems, reinforcement learning, and non-linear estimation and control. Currently, the group has activities in the areas of DES and Hybrid dynamical systems and in Reinforcement Learning. The presentation included a detailed description of how the group had addressed the points brought up in the previous (1998) advisory committee report. Most have been successfully addressed, however the area of interaction with other research groups both within the institute and outside still needs improvement. One talk was presented by a student on Hybrid Systems Abstraction and Composition. Applications included soccer playing robots, satellite formation control. 9

16 Overall the group uses Systems approaches to address problems typically drawn from the robotics and at the task coordination level. They are very familiar with the latest approaches in DES systems and the Ph.D. work of Paulo Tabuada is of high quality, theoretically very challenging work. It was rather surprising to see such highly theoretical work in a group where most of the other projects have a much more practical flavor. This group is proposing to merge with the AIMS group with one of the joint projects being the multi-robot project. I do like the work Pedro Lima is doing, but I think his efforts are being restricted by the fact that he is the only faculty in the group. Also a limiting factor in the group is the fact that there is no professional software (and hardware, but I do not think this is as severe) support, such as a systems specialist. Another issue, which I must mention is that I did not see participation in European projects among the group activities. AI and Manufacturing Systems (AIMS) Leader: Luis Custodio Associate Professor The Artificial Intelligence aspect of this group looks at emotion-based agents, multi-robotic agent systems, and spatial and pictorial reasoning. The Manufacturing Systems aspect of this group studies the control architectures for manufacturing systems. Professor Carlos Ferreira has not been able to participate in group research due to other commitments. The third member Carlos Bispo has research interests that are rather different from this group's interests. I met with Carlos individually to discuss his research and made recommendations regarding research focus areas. An overview of the activities was presented by Luis Custodio and then individual students presented summaries of their projects. There are two main research areas currently pursued, one that involves Emotion based Agent Systems and another that concentrates on issues regarding multi-agent systems with emphasis on the emotion based area. It is an active group with many conference publications and some journal publications. It seems to be one of the primary groups in emotions AI research internationally. This group is proposing to merge with the AIMS group with one of the joint projects being the multi-robot project. I could make similar comments as for the IC group, namely: I think the efforts of this group are being restricted by the fact that there is only one active faculty in the group. Also a limiting factor in the group is the fact that there is no professional software (and hardware, but I do not think this is as severe) support, such as a systems specialist. I did not see participation in European projects. Regarding both groups IC and AIMS and the proposed merger: There is no critical mass of senior researchers in either of the groups, there are too many different projects for the number of faculty, there is not much time for cooperation with other groups, for looking up new research directions and for building up research teams for European level proposals (such activities not only bring in resources but also provide visibility and a forum for the work at ISR to be measured against related work around the world). I do find the idea of merging the two groups overall very attractive. Even in this case however, the new group may not have critical mass in terms of number of active faculty and I would recommend that additional faculty be added. Another way is of course to reduce the teaching loads of these people. Ideally both should be done. I think that it is very important to give access of these groups to a technician and a software specialist. Such person may be shared by several groups. 10

17 Regarding projects where both groups may work on, it has been suggested that a multi-agent robot project is a candidate. I would like to support this suggestion but enhance the problem by considering larger number of robot agents, with hard real-time constraints and perhaps limited communication abilities. Such systems (networked, embedded) are becoming increasingly important in a wide range of applications. Evolutionary Systems and Biomedical Engineering: Leader: Agostinho Rosa Associate Professor This group focuses its research on the development of algorithms with gene oriented paradigms and gene oriented crossover and mutation operators. This is an active group with international partnerships. The fact that there is only one faculty in the group is a drawback. The group has significant expertise in EEG research and research in this area should be continued and should be emphasized. Smaller projects such as development of algorithms based on Cultural Information and social networks, although interesting they should be conducted in the background. This applies also to the project on the Olive Fly. Emphasize strength areas (EEG). The JAVA library of evolutionary algorithms the group has developed and made available is quite impressive. Recommendations include exploring research possibilities in the area of bioinformatics, focusing the research, reduce the number of MS projects if they distract from main research activities, and go after EU projects that provide access to major resources and increased visibility. Dynamical Systems and Ocean Robotics: Leader: Antonio Pascoal-Associate Professor This is a very successful and well-managed group with strong international reputation. They work on great topics combining theoretical research with exciting applications in the area of ocean robotics, the leadership is first class, their presentations were very well prepared and impressive. This group is a success story and its work should be supported as much as possible. Their work is an excellent combination of theory and practice in a topic very appropriate for Portugal and her marine traditions and should serve as a model for the other groups. The group has designed and constructed autonomous underwater vehicles and has developed advanced systems for navigation, guidance and control, acoustic communications, and mission management. Current projects include the study of hydrothermal phenomena and the mapping of marine habitats. Projects include the cooperation between a surface vessel and an underwater vehicle and involve research in signal processing, guidance and control and underwater communications. It is apparent that this group can use some relief from underwater vehicle construction and maintenance so to focus further on more research oriented issues. Aeronautics: Leader: Luis Campos Full Professor This group looks at flight testing, flight dynamics, flight simulation, aerodynamics, sound generation, sound propagation, and hydro magnetic waves. 11

18 Currently most projects are in acoustics and they include sound propagation in nozzles, blade vortex interaction, sound generation by propellers and acoustic fatigue. They are mostly theoretical projects that involve modeling and analysis. There are additional projects in flight dynamics that involve the separation of aircraft while landing and in magneto hydrodynamics involving solar wind. The group is involved in European projects. I did not meet Professor Campos but I was given his resume. The group has a large number of papers that primarily appear to highlight Professor Campos s expertise. In all papers, Professor Campos s name appears first. I would strongly recommend that students should take a more leading role in research and present papers in international conferences. I was very disappointed by the fact that there is no attempt whatsoever to integrate this group's research with the research of the other groups in ISR. It is not clear what anybody gains by having this group part of ISR. There are very exciting new research areas, such as coordination of multiple AUV s, etc.. where expertise in flight dynamics would be helpful in joint projects. Unfortunately Professor Campos is very busy and successful with his current research in the analysis of acoustic phenomena to have time to explore new areas and ways to bring his group closer to the research directions the rest of the ISR groups pursue. My strong recommendation is either to drop this group from ISR or better perhaps, to bring in a second faculty member (senior researcher) whose aim will be to establish within one year common ground and submit a joint proposal with other ISR groups. 12

19 1.1.3 Report by Prof. Mos Kaveh OVERVIEW ISR is a dynamic center of research with considerable strengths in systems, signal processing and controls. The staff and the students at the Institute continue to make contributions in both the theoretical and applications side of the mentioned disciplines, with hardware and experimental work primarily focused on a variety of robotics applications, including underwater autonomous vehicles. The existence of experimental projects provides students, even those outside of robotics, a more balanced exposure to hardware and practical issues than is often the case in most academic electrical engineering control and signal processing laboratories. The Institute has a dedicated and active faculty, many outstanding students, and a fair and visionary leader. The Institute has implemented a number of suggestions that were made at the time of the last visit by the External Advisory Committee in These positive developments include the establishment of a Theory Group and the offering of advanced graduate courses in several key topics, the institution of a seminar series for student researchers and visitors, and the establishment of a PhD course requirement. There is, of course, mixed feelings on the part of the students regarding this new PhD course requirement. The students, nevertheless, would like to see a greater variety of courses, particularly in signal processing and communications. Also, the students prefer seminar talks that are more shorter with more time for discussion. Consideration should also be made to the establishment of a mechanism for early review of thesis proposals by a committee consisting of faculty from at least two laboratories. This can serve as a quality control measure, and as a process for cross-pollination of ideas and possible increase in collaborations among different laboratories. Several accomplished young faculty members have, or are about to join the Institute. The accomplishments and research promise of most of these faculty members compare favorably with their counterparts in first rate universities in Europe and the US. As in the case of their mentors at the ISR, these young faculty members are in great need of administrative support, reduced teaching loads and merit-based promotions to bring their scholarly promise to full fruition. The discussions that have commenced in this regard at the instigation of Prof. Athans is a positive first step toward possible implementation of policies that recognize outstanding research in addition to good teaching, and that will serve to move ISR to even higher levels of excellence and international recognition. This, in turn, should help attract greater number of international students and visitors to the Institute. The graduate students are highly appreciative of the accessibility of the research faculty. Nevertheless, they worry that too much administrative load reduces the availability of thesis advisors for direct student supervision. Students and faculty also continue to be concerned about the general lack of administrative and computer system support, and access to electronic journals. They acknowledge that some improvements have been made at the university in support of research contract administration, although management and coordination of major contracts still consume substantial faculty time. The bulk of the general administrative load for the Institute continues to be on the Director, although some changes, in the form of assignment of associate directorship has recently been implemented. The following gives a summary of the observations on the three laboratories that were visited. 13

20 COMPUTER VISION The Computer Vision Laboratory is led by a combination of highly accomplished senior and junior faculty members. The Lab is primarily concerned with vision-based systems and image processing for robotics applications. Within this framework, research on a set of theoretical issues, such as learning and control, and technological innovations for sensing, such as omni directional camera systems, and specially-designed mirrors, have been pursued. The Laboratory has developed a wide range of expertise and basic technological know-how that can be extended to other areas utilizing computer vision, for example in traffic management. The group is quite active, and disseminates its research findings in major international journals and conferences. The projects have and continue to be funded by European and Portuguese sources. Key projects in the Computer Vision Group can be classified as falling in the areas of: 1. Active vision and robot learning through imitation and development 2. Robot navigation with omni directional vision 3. Visual underwater navigation 4. 3-D reconstruction of structured scenes from multiple 2-D projections The Lab s more recent work in active vision has moved into methods based on log-polar image representation, biologically-inspired system models, imitation and learning. In this regard collaboration with psychologists and some level of interaction with the Evolutionary Systems and Bioengineering Laboratory might be of benefit. Vision-based navigation is heavily based on the development of omni directional cameras and mixed mirror sensors designed for focus on, and resolution of environmental features that are useful for navigation. These are complemented with work on the development of topological maps and information sampling. The work appears (to a non-specialist) to be of high quality. Given the broader interest in the integration of sensors and information processors, the approach that is followed here can serve as an example of interesting work in the future that goes beyond camera-based sensors for robot navigation. The work on visual underwater navigation (NARVAL) is concerned with the construction of a mosaic map of the sea floor to use for navigation and servoing. Inherent difficulties with a visual system in this medium include low visibility, non-stationary lighting and environment and computational burden. Field data has been obtained and processed, and an experiment has been conducted with French collaborators. Extensions to nonplanar sea bottom and design of the robot controller are planned. It appears that the basic techniques related to the use of mosaic registration for determining trajectories have applications beyond the underwater problem. It seems natural to expect some level of collaboration on this project with the Dynamical Systems and Ocean Robotics group. A number of approaches and applications are being pursued that are related to 3D reconstruction and motion estimation. Basic research issues are concerned with the use of occlusions for 3D structure and motion, polyhedral approximations of structured 3D scenes from multiple images and optimal reconstructions (maximum likelihood) and uncertainty analysis of the reconstructions. The work seems quite solid, and the examples and applications seem to be quite diverse. EVOLUTIONARY SYSTEMS AND BIOENGINEERING This is a rather large and diverse group that is led by one faculty member. The projects appear to have little in common. However, they are mostly biologically inspired, use evolutionary computational techniques (genetic algorithms), and utilize a computational toolset that has and continues to be developed within the group. This set of software, The Java Distributed Evolutionary Algorithms (JDEAL) is openly available through the Lab's web site, and has apparently received attention by many research groups. 14

21 The best-known sustained work of the lab is related to sleep EEG analysis. This work clearly has international visibility, and is supported by several national and international collaborations, such as one with the Stanford University Medical School. Other examples of the group's most recent, current, and future projects include: 1. Artificial life 2. Planning and scheduling 3. Object recognition from feature networks 4. Use of EEG for cognitive studies and recognition of emotions. 5. Image enhancement The object recognition project is nearly finished, EEG work is ongoing with applications to drowsiness, sleep and road accidents. The artificial life work is concerned with the simulation of an agricultural pest problem - the olive fly, while topic 4 is to be carried out in the future, given appropriate support. For its size and level of activity, the Lab is low on research funds. However, the topics are popular, attract a large number of students. Collaborations with hospitals and other entities for data acquisition and experimentation seem to be critical to the generation of scholarly results and theses. Indeed, the lab's conference and journal publication output in 2002 promises to be quite healthy, after two years of decline following the Lab Director s sabbatical. Observations and suggestions The Lab director is visionary and appears to have established an intellectually stimulating environment for work at the interface of several disciplines. In principle, such cross-disciplinary activities can be an important source of discovery, and should be encouraged. National and international collaborations are extensive and strong. Co-advising by advisors of students who have come to the Lab from other institutions is an interesting concept for strengthening cross-institutional links. The co-advisors, however, do not appear to be actively involved in the supervision of the projects. A number of the projects have matured and are much better defined than they were at the time of the Committee s previous visit. The creation of JDEAL, and its open dissemination, if successful, is a good source of visibility for the Lab and the Institute. Diversity of the projects and the size of the group make advising and quality control of theses challenging. Publications are in widely varying disciplines (medical, computing, neural nets). While the breadth of disciplinary coverage is admirable, it is more difficult to gain strong visibility for the Lab in a particular field. A number of graduate students in the group come from non-ee, computing or physics/math programs. Appropriate disciplinary underpinnings must be provided to these students for a PhD degree in electrical engineering. More interaction with other groups within ISR would be desirable. Examples of possible projects that could benefit from such interactions include work in computational biology and computer vision and image processing based on the human visual system. Students from the Lab should be encouraged to give presentations on their work at the Institute seminar series. SIGNAL AND IMAGE PROCESSING The Signal Processing Laboratory has the largest number of faculty members of any Laboratory within ISR (soon to be nine faculty members in Lisboa, and two at Algarve). With one possible exception (Prof. Simoes?) all the faculty members have been active in research of high international quality in the past three years. A large number of publications and presentations have appeared in leading journals and conferences of the field. Professors Lourtie and Barroso and dubuf have had associate editorial responsibilities within the IEEE Signal Processing Society and the Int. Journal. of Pattern Recog. and AI, respectively, and Prof. Barroso continues to be highly visible within the signal processing community in the US. For example, in addition to serving on a technical committee of the IEEE Signal Processing Society, he has served on a proposal evaluation panel of the US National Science Foundation. The new group of assistant professors have superb research credentials that should serve the Laboratory well in the coming years. The Lab continues to have a strong funding base. For 15

22 example, special programmatic funding has been allocated for Signal Processing for Wireless, and the Diatom project coordinated by Prof. dubuf at Algarve. The primary projects in the Lab can be classified as: 1. Signal processing for robotics applications 2. Underwater acoustics and communications 3. Image processing and pattern recognition 4. Statistical signal and array processing, and signal processing for wireless The Signal Processing Lab has had a number of fruitful collaborations with other laboratories of the Institute that are primarily concerned with robotics applications. Given the substantial effort within the institute on various robotics and autonomous vehicle projects, this collaboration can continue to be a source of interesting signal processing projects that utilize and benefit real working systems. The Lab (at Lisboa and Algarve) has considerable strength in the area of underwater acoustics, signal processing and communications. Time-frequency analysis techniques and underwater acoustic topography are hallmarks of the techniques developed by the members of the Laboratory. The work is of outstanding quality, collaborations within and outside the Lab are strong, and the Lab appears to enjoy considerable visibility within the underwater acoustics community. Planned projects related to deep water topography and ocean environmental modeling based on acoustic inversion seem very challenging and interesting. The underwater acoustics and signal processing work at Algarve seems to have considerable level of synergy with the activities in Lisboa. Image processing and pattern recognition activities in the Lab include ultrasound imaging and shape tracking based in Lisboa ( Prof. Marques), and visual perception, pattern recognition and identification of diatoms (unicellular algae) at Algarve (Prof. dubuf). The work appears to be of high quality and with specific applications. The Algarve leadership of the Diatom project is noteworthy. The ultrasound work is concerned with 3D reconstruction in the presence of nonlinear compression in ultrasound scanners using statistical techniques. This is interesting work, and provides nice potential avenues of interaction with medical applications. However, there is considerable work done in ultrasound imaging for dealing with speckle, for example by using second harmonic filtering. It is not clear how the work being carried out in the Lab compares with the state of the art. The work on shape tracking based on multiple switched models is also very interesting. Given other activities within the Institute related to computer vision, and biomedical applications, the image processing work can benefit from cross-laboratory collaborations. However, the vision work at Algarve seems less synergistic with the activities in Lisboa, and its relation to ISR is not as clear. The Lab has established itself as an outstanding center of research in signal processing for communications with considerable international visibility. This is a direct result of the group s contributions to the general area of statistical signal and array processing. The regional/local needs and opportunities have resulted in key contributions to underwater signal processing and communications. This has motivated the understanding of some difficult channel models, and the development of algorithms to deal with limitations of such channels. At a more basic general level, the group has contributed scholarly work to blind identification of channels, including MIMO systems that are gaining considerable interest in wireless communications. The special funding for signal processing for wireless, and the incoming staff members should boost the group s activities in this area. Related to this work, the ISR group has the opportunity to more closely interact with the wireless and microwave communication systems groups in the Telecommunications Institute of IST. While some level of interaction exists, the potential for more significant collaboration is great. I visited the microwave laboratory. The equipment and expertise in this lab can provide an experimental infrastructure for channel modeling and signal processing for wireless that is seldom found in leading academic signal processing groups. 16

23 2. RESEARCH TEAM AND INTERESTS 2.1 MEMBERS AND COLLABORATOR THEORY GROUP: Michael ATHANS, Principal Researcher ARTIFICIAL INTELLIGENCE AND MANUFACT. SYSTEMS: Carlos PINTO-FERREIRA, Associate Professor Luis M. CUSTÓDIO, Assistant Professor Carlos BISPO, Assistant Professor Sandra GADANHO, Post-Doc Jorge Miguel PAIS, Adj. Professor, Ph.D. St. Rodrigo VENTURA, Teaching Assistant, Ph.D St. Nuno ORFÃO, Teaching Assistant, Ph.D St. Márcia MAÇÃS, M.Sc. St. Ivone FERNANDES, M.Sc. St Bruno DAMAS, M.Sc. St. Pedro VALE, M.Sc. St Cristian MUNTEANU, Ph.D. St. José MALAQUIAS, Ph.D. St., Teaching Assistant Ernesto SOARES, Ph. D. St. João Paulo CALDEIRA, M.Sc. St., Teaching Assistant Rui TAVARES, M.Sc. St., Teaching Assistant Carlos FERNANDES, M.Sc. St. Nelson PEREIRA, M.Sc. St. Henrique PEREIRA, Research Project Weng Hong TANG, Undergrad. St. Alexandre CALAPEZ, Undergrad. St.- BIC Sergio CHIN, Undergrad St. Djamilo JACINTO, Undergrad St. Atila NEVES, Undergrad. St Miguel COSTA, Undergrad St. BIC Ivo BHATT, Undergrad. St Andre BALTAZAR, Undergrad. St Pedro SOEIMA, Undergrad. St José Carlos REIS, Undergrad. St Sara ALI, Undergrad. St Nuno COSTA, Undergrad. St Fernando CONTREIRAS, Undergrad. St Marco ERRA, Undergrad. St Pedro MITRA, Undergrad. St Nidia CALDEIRA, secretary COMPUTER AND ROBOT VISION : João SENTIEIRO, Full Professor, ISR/IST Director José SANTOS-VICTOR, Assistant Professor João Paulo S. A. COSTEIRA, Assistant Professor Alexandre BERNARDINO,Teach.Assist.,Ph.D. St. José GASPAR, Teaching Assistant, Ph.D. St. Nuno GRACIAS, Ph.D. St. Etienne GROSSMAN, Ph.D. St. João MACIEL, Ph.D. St. César SILVA, Ph.D. St. Niall WINTERS, Ph.D. St. Raquel VASSALO, Ph.D. St. Claudia DECCÒ, Ph.D. St. Lenildo SILVA, Ph.D. St. Freek STULP, Ph.D. St. Manuel LOPES, Ph.D. St. Sjoerd van der ZWAAN, M.Sc. Researcher INTELLIGENT CONTROL : Pedro LIMA, Assistant Professor (IST) Paulo TABUADA, Ph.D. Student Dejan MILUTINOVIC, Ph.D. Student Sónia MARQUES, Ph.D. Student, Teaching Assistant (IPS) Paulo ALVITO, M.Sc. Student, Teaching Assistant (IPS) João PINA, M.Sc. Student, Teaching Assistant (FCT/UNL) Carlos MARQUES, M.Sc. Student Hugo FURTADO, M.Sc. Student João FRAZÃO, Undergrad St Paulo SARGENTO, Undergrad. St Hugo COSTELHA, Undergrad. St Gonçalo NETO, Undergrad. St Pedro PINHEIRO, Undergrad. St Duarte SEGURADO, Undergrad. St Carlos SANTOS, Undergrad. St Pedro PEREIRA, Undergrad. St Victor GOMES, Undergrad. St EVOLUTIONARY SYSTEMS AND BIOMEDICAL ENG. : Agostinho ROSA, Associate Professor Fernando MELÍCIO, Adj. Professor, Ph.D. St. Rogério LARGO, Adj. Professor, Ph.D. St. Tito SILVA, Ph.D. St. Osvaldo BRASAO, Ph. D. St. Hongfei GONG, Ph.D. St. 17

24 MOBILE ROBOTICS : Maria Isabel RIBEIRO, Associate Professor João SEQUEIRA, Assistant Professor Alberto VALE, Ph.D. Student José Paulo CASTRO, M.Sc. Student João G. MOTA, M.Sc. Student Sérgio GUERREIRO, M.Sc. Student Inácio ROCHA, M.Sc. Student Nuno SILVA, Undergrad. St Nelson GONÇALVES, Undergrad. St Pedro SILVA, Undergrad. St SIGNAL PROCESSING : (IST) Victor BARROSO, Associate Professor Isabel LOURTIE, Associate Professor Jorge S. MARQUES, Associate Professor Pedro AGUIAR, Assistant Professor Francisco GARCIA, Assistant Professor João Pedro GOMES, Teach. Assistant, Ph.D. St. João XAVIER, Teach. Assistant, Ph.D. St. João SANCHES, Teach. Assistant, Ph.D. St. Jacinto NASCIMENTO, Ph.D. St. Paulo M. de OLIVEIRA, Ph.D. St. Jorge BARBOSA, Ph.D. St. Dusan RAMLJAK, Ph.D. St. Emanuel RIBEIRO, Ph.D. St. Pedro JORGE, Ph.D. St. Rui CRUZ, M.Sc. Student Paulo LOPES, Undergrad St. Tiago PATRÃO, Undergrad St. Rui F. C. GUERREIRO Undergrad. St DYNAMICAL SYSTEMS AND OCEAN ROBOTICS : António PASCOAL, Associate Professor Carlos SILVESTRE, Assistant Professor Sergey RUMYANTZEV, Post-Doc Leonel LAPIERRE, Post-Doc Didik SOETANTO, Post-Doc Paulo OLIVEIRA, Teach. Assistant, Ph.D. St. Pedro ENCARNAÇÃO, Ph.D. St. António AGUIAR, Ph.D. St. Francisco TEIXEIRA, PhD. St. Fei CHUN MA, Ph.D. St. Sajjad ASL, Ph.D. St. João ALVES, M.Sc. St. Carlos FERREIRA, M.Sc. St. Miguel PRADO, M.Sc. St. Luella WONG, M.Sc. St. Rita CUNHA, M.Sc. St. Pedro ALVES, M.Sc. St. Luis SEBASTIÃO, Res. Engineer Manuel RUFINO, Res. Engineer Telmo AZEVEDO, Research Technician Nuno FERRAZ, Undergrad. St. Álvaro PIRES, Undergrad. St. Paulo SANTOS, Undergrad. St. Rui SIMÕES, Undergrad. St. ADMINISTRATIVE STAFF: Filomena VIEGAS Loic BANDÉ Nuno SENA SIGNAL PROCESSING : (U.ALGARVE) Sérgio JESUS, Associate Professor Hans du BUF, Invited Associate Professor Hamid SHAHBAZKIA, Assistant Professor Orlando C. RODRIGUEZ, Assistant Professor Mário C. M. JESUS, Adjoint Professor António João SILVA, Adjoin Professor, Ph.D. St. Paulo FELISBERTO, Adjoin Professor, Ph.D. St. Robert LOKE, Ph.D. Student João RODRIGUES, Ph.D. Student Luis SANTOS, Ph.D. Student Pedro GUERREIRO, Ph. D St. R. LAM, Ph. D St. Cristiano SOARES, M.Sc. St. Nelson MARTINS, M.Sc. St Bruno SILVA, Undergrad. St. AERONAUTICS : Luís M. Braga da Costa CAMPOS, Full Professor Fernando P. LAU, Assistant Professor Paulo J. S. GIL, Assistant Professor Filipe S.R.P. CUNHA, Invited Professor António J.N.M. AGUIAR, Invited Assistant Jorge M.P. GOMES, Assistant Pedro G.T.A. SERRÃO, Teaching Assistant, Ph.D. Student João M. G. OLIVEIRA, Teaching Assistant, Ph.D. Student Pedro M. V. MIRANDA MENDES, Ph.D. Researcher Mª Fernanda VENÂNCIO, Secretary Ana D. MORGADO, Secretary Sónia Varela BORGES, Secretary 18

25 2.2 CURRENT RESEARCH INTERESTS The Lisbon pole of ISR is internally organized in 8 laboratories/groups. In this section the main research interests of each one of the Laboratories/groups are briefly described INTELLIGENT CONTROL (IC) The main research problems pursued at the Intelligent Control Lab are: the control of complex and/or large-scale systems, the implementation of real-time process and (cooperative or individual) robot control architectures encompassing low-level control, sub-systems coordination and task planning. The research topics include: Hybrid Systems: study of control systems composed of a supervision/coordination sub-system (event-driven) and one or more sub-systems which interact directly with the controlled system/environment. The group is currently more interested in the applications of hybrid systems to the performance analysis and synthesis (from specifications) of robotic tasks, especially through the use of Petri nets and hybrid automata. Multi-Robot Systems: analysis and synthesis of a population of robots designed to achieve specific goal(s) through cooperation between the population members. The group is currently interested in the application to Robotic Soccer, where the environment is full of static (walls, stopped robots) and dynamic objects, including repulsive (other robots) and attractive dynamic objects (the ball), as well as to Robotic Rescue in situations of large-scale disasters (e.g., earthquakes, building collapse). Reinforcement Learning: this is a specific type of learning appropriate for applications to robotics, with potential, to be linked to the work on Hybrid Systems and Multi-Robot Systems. Non-Linear Estimation and Control: this topic is mainly motivated by the group work on satellite attitude estimation and control. Also mobile robot guidance requires considerable knowledge from this area. 19

26 20

27 2.2.2 COMPUTER AND ROBOT VISION (VIS) The research conducted in the Vislab is organized in two main lines: Vision Based Control and Navigation 3D Motion analysis and Reconstruction When a camera is moving in a static (or dynamic) environment, the image sequence conveys information regarding the scene/objects structure and camera/objects motion. In the theme of Vision Based Control and Navigation, we address the fundamental problems of understanding what relevant information can be extracted from the image sequences to control a robot in order to perform a given task. This has long standing research line of the Vislab including the control of extremely varied systems like the active control of a binocular head, vision based navigation for land, aerial and underwater vehicles and teleoperated cellular robots. From the foundation, VisLab has addressed the control of Binocular Systems and a land mobile robots. More recently, the research was extended to vehicles moving in the 3D space, such that the interplay between the vehicle s degrees of freedom and the scene structure is significantly richer. Example applications have focused on lighter than air blimps and underwater vehicles, in the context of European Research Projects. One distinctive aspect is the search for alternative imaging geometries, often inspired after biological findings. Extensive work has been carried out in the domain of using omni directional images for navigation and control as well as for map building for indoors robots. In addition, we have pursued the usage of non-metric maps for navigation like topological maps for structured environments and video mosaics for underwater navigation. 21

28 The theme of 3D motion Analysis and Reconstruction is devoted to the geometry of retrieving information about the scene structure or camera motion from video sequences. Work has addressed the problem of estimating the 3D motion of a camera from an image sequence. Several visual cues were exploited for this purpose: the visual motion and occlusions. Regarding 3D reconstruction, work has focused on developing optimal approaches for matching image features, which is a key step in most 3D vision systems. In addition, the depth estimation process has been formulated in an optimal way by itself. Another line of research has been the reconstruction of structured scenes (e.g. buildings) making use of auxiliary geometric information provided by the user. Research in all these topics has ben carried out both at at the level of the fundamental methodologies and also for applications. As the knowledge in these various aspects matures inside the group, research projects have been proposed, including national and European Projects. 22

29 2.2.3 ARTIFICIAL INTELIGENCE AND MANUFACTURING SYSTEMS (AIMS) The research conducted in the AIMSlab is organized in two main lines: Emotion-based Agent Systems Multi (Robotic) Agent Systems Emotion-based Agent Systems Recent research on the neurophysiology of human emotions suggests that human decision-making efficiency depends deeply on the emotions mechanism.eneral methodology of this research consists of the study of the neuroscience findings (e.g., Damásio and LeDoux works) and research how these results/theories/hypothesis might be used/interpreted/adapted for designing better or different (artificial intelligence) agents. The problem is not whether intelligent machines can have emotions, but whether machines can be intelligent without any emotions [Marvin Minsky, The Society of Mind] development of an emotion-based agent architectures, which is an entity whose behaviors is designed through a conceptual interpretation/implementation of the hypothesis/theories formulated in the neuroscience area, related with human emotions, namely the concepts of primary and secondary emotions, somatic marker hypothesis, stimuli double and parallel processing, conditioning, movie-in-the-brain. The research methodology is based on experimentation either with virtual or real environments study how an emotion-based architecture might be articulated with a classical rational-based architecture (which includes knowledge representation, reasoning, learning, planning, to name but a few).aking a known (classical) tool and integrate it with our architecture, for instance, Logics (reasoning), Reinforcement Learning (learning), STRIPS (planning). formalize the conceptual framework developed in order to have a formal modelling tool for analysing and synthesizing emotion-based agents (for instance, using Theory of Categories, Drama Theory (a re-definition of Game Theory that assumes non completely rational decision making). Multi (Robotic) Agent Systems In the context of Cooperative Robotics, in order to obtain useful cooperation, relationships among robots have to be accomplished, because of the existence of dependencies between agents actions, the definition of global constraints and goals, and the fact that each individual might have insufficient competence, resources and information to solve the problem per se. So, the goal of this research is the development of general methodologies for teamwork in a multi-agent system, specially for multi-robot system, capable of dealing with complex and dynamic environments, working coherently as a group of agents, handling different and even opposite views of the world and problem, allowing flexible communication among team members, evaluating the team performance, and implementing re-organization strategies to handle unexpected situations. 23

30 define the underlying individual agent architecture, specially both individual and cooperative knowledge (or belief) representation, (e.g., Logic-based, behaviour-based, hierarchical-based, Belief-Desire-Intention); develop a formal reasoning (or belief revision) mechanism for updating knowledge (or belief), establishing and terminating cooperative actions, evaluating its pre-conditions, choosing which agents participate and deciding which agents might have the initiative to establish cooperation, for instance, based on First Order Logic (e.g., Joint Intentions Theory), Modal Logics (e.g., LORE logic), True-Maintenance System, Bayesian inference, Game Theory, Contract Nets; design a planning and scheduling tools of cooperative actions for distributing them among the agents handling limited resources, and re-planning in order to recover from failures, based on Distributed Planning (e.g., Shared Plans Theory), Distributed Constrained Heuristic Search 24

31 2.2.4 MOBILE ROBOTICS (MR) The Mobile Robotics Lab activities focuses on the research, development and testing of robotic tools applied to the control and navigation of autonomous mobile robotis. In this Laboratory we are particularly interested in the issues of: mobile robot navigation, in structured and semi-structured environments, cooperation/collaboration among multiple robotic devices, robotics and information systems. Mobile robot navigation: Study of navigation architectures for the operation of mobile robots in structure and semistructured indoor environments, including obstacle detection and avoidance, path planning, trajectory finding, motion control and localization. Different sensors are used, namely ultrasound and laser. The group is most interested in the establishment of new sensor and world representations aiming at simplifying the navigation tasks, namely to overcome the absolute localization required in most tasks. The study of probabilistic approaches for the Localization and Map Building in outdoors environment aiming at search and rescue operations is currently under study. Cooperative robotics: Multiple heterogeneous robots (mobile platforms and manipulators) acting together towards the fulfilment of an assigned task. A behaviour-based approach to the control of each single robot has been considered. Formal aspects of the behaviour-based control architectures were studied using tools from algebraic group theory. These led to a conceptual control architecture of hybrid nature, with a supervisor modelled by a finite discrete automaton and a set of classes of continuous models modelling robot motion. A distinctive feature of these continuous models is that they accept (in the sense that an assigned mission can be successfully executed) a broad range of robot trajectories. Robotics and information systems: Information systems are one of the cornerstones of most of the modern organizations. Furthermore, the use of CASE tools in organizations management/operation led to the development of abstract modelling languages of which one of the most widely used is UML (Universal Modelling Language). The biological inspiration has been used in many areas of robotics, such as sensors and robot control architectures. Furthermore, the recent explosion of cooperative robotics is also absorbing paradigms from social evolution models to minimize the complexity of the problem. A similar approach was followed to design a robot control architecture based on a business modelling framework. Unlike the classical approach, this robot control architecture is defined for each mission assigned to the robot and it is revised each time an event in a pre-specified set is triggered. It is also expected that the overall methodology can be applied to robot teams. 25

32 2.2.5 SIGNAL AND IMAGE PROCESSING (SP) Statistical Array and Signal Processing Wireless Communication Systems Underwater Acoustic Data Communications Detection and Estimation Theory Time-Frequency Signal Analysis and Processing Navigation and Guidance of underwater vehicles Image Processing shape analysis video surveillance pattern recognition Image analysis Computer vision Video processing Multimedia signal processing EVOLUTIONARY SYSTEMS AND BIOMEDICAL ENGINEERING (ESBE) In the last three years a great effort has been directed to the development of the new areas of Evolutionary Computation and Computational Intelligence. An application oriented approach has been the early focus and will remain so, but recently the attention has been more directed towards methodology and paradigm areas. Current research in this new objectives are: Development of algorithms with gene oriented paradigms instead of individual or chromosome centered. New gene oriented crossover and mutation operators (infection operators) have been devised and applied; more applications and general conceptual framework undergoing. The symbiotic/synergetic use of Evolutionary Computation with Artificial Life simulation models (as insect colony optimization, disease simulation and prediction, etc). The initial steps for the application of Evolutionary Computation in the area of Computational Molecular Biology will be taken. On the Biomedical Engineering area, the core work on sleep research will proceed with international cooperation. The area of cognitive science linked to the EEG as brain computer interface will be the next point of further exploration. 26

33 Gaia - Virtual Reality Artificial Life Simulator School Timetable Scheduling using Genetic Algorithms 27

34 Search Problems Solving unsing Evolutionary Search Integrated Protection of Crops Using Artificial Life Simulation 28

35 2.2.7 DYNAMIC SYSTEMS AND OCEAN ROBOTICS (DSOR) The research and development work carried out at the Dynamical Systems and Ocean Robotics Laboratory (DSORlab) of ISR aims at contributing to furthering the knowledge in the general area of dynamical system theory and applying newly developed analysis and design tools to the control and operation of robotic ocean and air vehicles. Over the past the few years, research work has been focused on the study of advanced linear and nonlinear system theory and their use in the development of new methods for autonomous vehicle navigation, guidance, and control. Considerable effort has also been placed on the study of hybrid systems for mission control of robotic vehicles, to enable the analysis and design of entities that capture the interplay between time-driven and event-driven systems. The development work has led to the construction of the robotic ocean vehicles DELFIM (an autonomous surface Catamaran), INFANTE (an autonomous underwater robot), and CARAVELA (an autonomous oceanographic vessel). These vehicles play the dual role of i) advanced testbeds, to field test new system theoretical concepts, and ii) platforms for actual operations at sea, effectively paving the way for a fruitful symbiosis between marine science and technology. This follows the successful development of MARIUS, the first civilian European autonomous underwater vehicle for coastal oceanography in the scope of a project coordinated by ISR/IST, under the auspices of the Commission of the European Communities. The DSORL has also played an active role in the design, implementation, and at sea testing of the navigation, guidance, control, and mission control systems of SIRENE, an underwater shuttle for the automatic deployment of benthic laboratories developed in the scope of a European project coordinated by IFREMER, France. Currently, the DSORL is involved in a number of projects and concerted actions with national and foreign institutions with the objective of advancing engineering methodologies and equipments to the point where they can be used as versatile tools to expand our understanding of the oceans. At a technological level, this concerted effort is in line with the current trend worldwide, aimed at the development of ocean sampling networks (OSN) providing a nested ocean observation capability through the coordinated control of many, mobile, networked, sensor platforms. This trend shows clearly that advancements in marine robotics, communications, and information systems are steadily being brought to bear on the development of technologies to enable safer, better, faster, and far more efficient methodologies for the study of the oceans. At the same time, the plethora of engineering problems that must be tackled and solved in the context of ocean research pose considerable challenges to theoreticians and system designers. The CARAVELA Auto Ocean Vessel 1:3 scaled model The DELFIM ASC and INFANTE AUV 1:1 models 29

36 Recently, and as a natural consequence of a longstanding collaboration program with the Department of Aeronautics and Astronautics of the Naval Postgraduate School of Monterey, California, USA, the DSORL has started to apply some of the methods and technologies developed for ocean vehicles to the control of air robots. Again, the activity pursued in the area is well rooted in scientific applications that require the use of autonomous air robots to accurately map coastal areas subjected to erosion. At a theoretical level, the main lines of research that are being pursued at the DSORL are the following: Control of Nonholonomic and Underactuated Vehicles : stabilization of underactuated ocean robots in the presence of unknown ocean currents, actuator saturation, and vehicle parameter uncertainty using Lyapunov-based techniques, back stepping, and switched hybrid control. Trajectory Tracking and Path Following Control : development of linear and nonlinear control strategies for accurate trajectory tracking /path following. Linear designs build on gain-scheduled control theory and exploit the use of Linear Matrix Inequalities. Nonlinear designs resort to back stepping techniques and explicitly address the problems of parameter uncertainty. Considerable emphasis has been placed on the development of innovative methods that combine some of the desirable characteristics of both trajectory tracking and path following, i.e. time convergence to the trajectory, while maintaining the smooth behavior during the initial approach to that trajectory that is normally associated with path following. Work is also being done towards the development of sensor-based path following control laws for ocean vehicles using vision and sonar. In the air robotics area, work has progressed on the development of an accurate helicopter mathematical model for effective control system design and flight envelope expansion. The dynamic modeling focused on two-bladed single main rotor helicopters, and included a mathematical description of Bell-Hiller servo-rotors. The helicopter was modeled as a six degrees of freedom rigid body, actuated by forces and moments, which are generated at the main and tail rotors, fuselage, and empennage. Models for different flight regimes will be validated against the full nonlinear helicopter simulation model, providing the working ground for future system identification and control design / implementation for an existing helicopter that has been fully instrumented. Coordinated Vehicle Control : multiple vehicle control for the execution of joint missions at sea. The main practical motivation for this work stems from the need to operate an autonomous surface craft (ASC) and an underwater vehicle (AUV) at the same time, along the same projected horizontal path, while keeping the two vehicles aligned along a vertical line (ASIMOV concept). See the figure below, where the AUV plays the role of master and the ASC that of slave in charge of relaying acoustic transmission from the AUV to a support ship. See also the following pictures that capture two phases of a mission exercise in the Azores, during the Summer of 2001, where the ARIES AUV of the Naval Postgraduate School of Monterey submerged, stabilized at a desired depth, and ran along a path identical to that of the DELFIM ASC, while communicating with it via an acoustic modem. The theoretical solutions adopted bear some affinity with those adopted for combined trajectory tracking and path following. At this stage, however, they do require the exchange of a large amount of information between the two platforms. Future work must address the problem of coordinated vehicle control under severe bandwidth constraints. 30 Coordinated vehicle operation: the ASIMOV concept

37 Joint operation of the ARIES AUV and DELFIM ASC in the Azores, Summer of 2001 Navigation System Design: new methods for navigation system design are being developed using the theory of multirate, polytopic, and linear parametrically varying systems. The main goal is to develop methodologies that can afford system designers with frequency-like design / analysis tools, thus extending to the time-varying and nonlinear setting the highly practical and intuitively appealing complementary filtering structures. The research activity in this area witnessed also the development of vision-based nonlinear complementary filtering structures for air and undersea applications. Control of Ocean Vessels in the Presence of Waves: the problem addressed is that of designing acceleration feedback control systems for precise maneuvering and wave disturbance attenuation of hydrofoil craft. The design methodologies explore simple key physical relationships and build on gain scheduling control techniques. Mission Control Systems (MCSs) : at the mission control level, work has continued on the development of software and hardware tools for mission programming and mission execution of autonomous vehicles, including cooperative control of surface and underwater vehicles. This work was instrumental in enhancing the capabilities of the Petri-net based software application named CORAL, proprietary of ISR/IST. At the same time, hardware architectures were developed for distributed real-time control of ocean robotic vehicles. Intensive series of tests at sea showed the reliability of the overall Mission Control System (MCS) developed. Namely, during the Summer of 2001, the DELFIM ASC successfully completed a series of seabed map building missions totaling up to more than 40 hours of operation in a purely autonomous mode, under the supervision of its MSC. During these missions the vehicle performed a grid survey, executing path following maneuvers in the presence of shifting sea currents and wind, while collecting acoustic data from a sidescan and mechanically scanned pencil beam sonar. The data were later processed to obtain high resolution seabed maps. 31