UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD APPLE INC., Petitioner v. IMMERSION CORPORATION, Patent Owner U.S.

Transcription

1 UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD, Petitioner v. IMMERSION CORPORATION, Patent Owner U.S. Patent No. 7,808,488 Filing Date: March 29, 2007 Issue Date: October 5, 2010 Title: Method and Apparatus For Providing Tactile Sensations Inter Partes Review No.: (Unassigned) DECLARATION OF DR. MAJID SARRAFZADEH Ex Page 1

2 TABLE OF CONTENTS Page I. INTRODUCTION... 1 A. Background and Qualifications... 1 B. Information Considered... 3 II. LEGAL STANDARDS... 4 A. Legal Standards for Prior Art... 4 B. Legal Standards for Anticipation... 5 C. Legal Standards for Obviousness... 6 D. Legal Standards for Priority III. OVERVIEW OF THE 488 PATENT A. Summary of the 488 Patent B. Claims Are Not Entitled to a Priority Date of November 1, C. Person of Ordinary Skill in the Art D. Claim Construction IV. PRIOR ART A. JP Limitation 9.pre: A system, comprising: Limitation 9.a: a touch-sensitive input device configured to output a sensor signal indicating an object contacting the touch-sensitive input device and a pressure of the contact Limitation 9.b: an actuator coupled to the touchsensitive input device, the actuator configured to receive an actuator signal and output a haptic effect to the touchsensitive surface based at least in part on the actuator signal, wherein the actuator signal is configured to cause a first haptic effect if the pressure is less than a first pressure threshold and cause a second haptic effect if the pressure is between the first pressure threshold and a second pressure threshold i Ex Page 2

3 TABLE OF CONTENTS (continued) Page 4. Limitation 9.c: a processor in communication with the sensor and the actuator, the processor configured to Limitation 9.d: [the processor configured to] output a display signal configured to display a graphical object on the touch-sensitive input device Limitation 9.e: [the processor configured to] receive the sensor signal from the touch-sensitive input device, Limitation 9.f: [the processor configured to] determine an interaction between the object contacting the touchsensitive surface and the graphical object based at least in part on the pressure Limitation 9.g: [the processor configured to] generate an actuator signal based at least in part on the interaction Limitation 9.h: [the processor configured to] transmit the actuator signal to the actuator Claim 11: The system of claim 9, wherein the processor is configured to output the actuator signal when the object contacts the touch-sensitive device at a location not corresponding to the graphical object Claim 12: The system of claim 9, wherein the display signal is configured to display a keypad comprising a plurality of softkeys Claim 13: The system of claim 12, wherein the haptic effect is caused to be output when a user contacts the touch-sensitive device at a location corresponding to a softkey in a home position Claim 14: The method of claim 12, wherein the plurality of softkeys comprises one softkey for each digit from 0 to Limitation 16.a: The method of claim 9, wherein the graphical object comprises a first graphical object and a second graphical object, and ii Ex Page 3

4 TABLE OF CONTENTS (continued) Page 15. Limitation 16.b: if the object contacts the first graphical object, the first haptic effect is configured to be output if the pressure is less than the first pressure threshold and the second haptic effect is configured to be output if the pressure is between the first pressure threshold and the second pressure threshold, and Limitation 16.c: if the object contacts the second graphical object a third haptic effect is configured to be output if the pressure is less than the first pressure threshold and a fourth haptic effect is configured to be output if the pressure is between the first pressure threshold and the second pressure threshold Limitation 1.pre: A method, comprising: Limitation 17.pre: A computer readable-medium comprising program code, comprising B. Tsuji and Roysden C. Tsuji and Gettemy D. Tsuji, Gettemy, and Westerman V. CONCLUSION iii Ex Page 4

5 Exhibit No. Description EXHIBIT LIST 1101 U.S. Patent No. 7,808,488 to Martin et al. ( 488 patent ) Declaration of expert Dr. Majid Sarrafzadeh ( Sarrafzadeh Decl. ) File history for U.S. Patent No. 7,808,488 ( 488 patent file history ) Patent Owner Immersion s claim chart regarding alleged infringement of the 488 patent by certain Apple iphone products (Exhibit 21 to Immersion s ITC Complaint in ITC Investigation No. 337-TA-1004) Patent Owner Immersion s second claim chart regarding alleged technical domestic industry for the 488 patent (Revised Exhibit 25 to Immersion s ITC Complaint in ITC Investigation No. 337-TA- 1004) Immersion s Complaint in ITC Investigation No. 337-TA-1004 (the Complaint ) Immersion s Preliminary Proposed Claim Constructions (-1004 Investigation Patents) Chief ALJ s Order No. 27 on Claim Construction in ITC Investigation No. 337-TA U.S. Patent No. 7,336,260 to Martin et al. ( Martin ) 1110 Japanese Published Application No. H Translation of Japanese Published Application No. H ( JP725 ) U.S. Patent No. 9,489,018 to Gettemy et al. ( Gettemy ) U.S. Patent No. 6,323,846 to Westerman et al. ( Westerman ) U.S. Patent No. 5,575,576 ( Roysden ) Provisional Application No. 60/335,493. iv Ex Page 5

6 I. INTRODUCTION 1. I have been retained by counsel for Apple Inc. as an expert witness in the above-captioned proceeding. I have been asked to provide my opinion about the patentability of claims 3-8, 11-16, and of U.S. Patent No. 7,336,488 (Ex. 1101) ( 488 patent ). I previously provided opinions regarding the patentability of claims 1, 2, 9, 10, 17, 18, and 29 of the 488 patent for IPR I have been retained at my normal hourly rate of $600 per hour. No part of my compensation is dependent upon the outcome of this proceeding or the specifics of my testimony. A. Background and Qualifications 3. My curriculum vitae, which includes a detailed record of my professional qualifications, including a list of publications, awards and honors, and professional activities, is attached as Exhibit A. Relevant highlights are summarized below. 4. I am a distinguished professor at the University of California at Los Angeles ( UCLA ), the director of the UCLA ehealth and Data Analytics Research Laboratory, and a co-director of the UCLA Center for SMART Health. I earned Bachelor of Science, Master of Science, and Ph.D. degrees from the University of Illinois at Urbana-Champaign in Electrical and Computer Engineering in 1982, 1984, and 1987, respectively. I became an Assistant 1 Ex Page 6

7 Professor at Northwestern University in 1987, earned tenure in 1993, and became a Full Professor in In 2000, I joined the Computer Science Department at UCLA as a Full Professor. In 2008, I co-founded and became a co-director of the UCLA Wireless Health Institute. And in 2016 I co-founded and became a codirector of the Center for Small health. 5. I have experience as a system designer, circuit designer, and software designer. This experience includes positions as a design engineer at IBM and Motorola and a test engineer at Central Data Corporation. I was the main architect of an Electronic Design Automation ( EDA ) software tool for Monterey Design Systems, Inc. ( Monterey ). Monterey was acquired by Synopsys in I cofounded and managed the technical team at Hierarchical Design, Inc. ( Hier Design ), an EDA company that specialized in reconfigurable FPGA systems. Hier Design was acquired by Xilinx in I have cofounded MediSens Wireless, Bruin Biometrics, and WANDA Health. 6. I have experience with Human Computer Interaction through design related to exercise gaming (exergaming) and mobile health (mhealth). I have designed systems wearable on foot and other parts of the body that facilitated interactions with games such as Soccer (FIFA), Guitar Hero, and Dance Dance Revolution, to name a few. I have also designed systems that patients with chronic 2 Ex Page 7

8 illness (e.g., heart failure) to interact with a communication device such as a smart phone or a smart watch. 7. I am a Fellow of the Institute of Electrical and Electronics Engineers, Inc. ( IEEE ) for my contributions to Theory and Practice of VLSI Design. I have served on the technical program committees of numerous conferences in the area of system design. I cofounded the International conference on Wireless Health and have served in various committees of this conference. 8. I have published approximately 545 papers, and have received a number of best paper and distinguished paper awards. I am a co-author of the book Synthesis Techniques and Optimizations for Reconfigurable Systems (2003 by Springer), a co-author of the book An Introduction to VLSI Physical Design (1996 by McGraw Hill), and a co-editor of Algorithmic Aspects of VLSI Layout (1993 by World Scientific Publishing). 9. Additional details regarding my qualifications and background can be found in my CV in Exhibit A at the end of this document. B. Information Considered 10. My opinions are based on my years of education, research, and experience, as well as my study of relevant materials. In forming my opinions, I have considered the materials identified in this declaration and in the Petition. 3 Ex Page 8

9 11. I may rely upon these materials and/or additional materials to respond to arguments raised by Patent Owner. I may also consider additional documents and information in forming any necessary opinions, including documents that may have not yet been provided to me. 12. My analysis of the materials produced in this proceeding is ongoing and I will continue to review any new material as it is provided. This declaration represents only those opinions I have formed to date. I reserve the right to revise, supplement, or amend my opinions stated herein based on new information and on my continuing analysis of the materials already provided. II. LEGAL STANDARDS A. Legal Standards for Prior Art 13. I understand that a patent or other publication must first qualify as prior art before it can be used to invalidate a patent claim. 14. I understand that a U.S. or foreign patent qualifies as prior art to an asserted patent if the date of issuance of the patent is prior to the invention of the asserted patent. I further understand that a printed publication, such as an article published in a magazine or trade publication, qualifies as prior art to an asserted patent if the date of publication is prior to the invention of the asserted patent. 15. I understand that a U.S. or foreign patent also qualifies as prior art to an asserted patent if the date of issuance of the patent is more than one year before 4 Ex Page 9

10 the filing date of the asserted patent. I further understand that a printed publication, such as an article published in a magazine or trade publication, constitutes prior art to an asserted patent if the publication occurs more than one year before the filing date of the asserted patent. 16. I understand that a U.S. patent qualifies as prior art to the asserted patent if the application for that patent was filed in the United Stated before the invention of the asserted patent. B. Legal Standards for Anticipation 17. I understand that documents and materials that qualify as prior art can be used to invalidate a patent claim via anticipation or obviousness. 18. I understand that, once the claims of a patent have been properly construed, the second step in determining anticipation of a patent claim requires a comparison of the properly construed claim language to the prior art on a limitation-by-limitation basis. 19. I understand that a prior art reference anticipates an asserted claim, and thus renders the claim invalid, if all elements of the claim are disclosed in that prior art reference, either explicitly or inherently (i.e., necessarily present). 20. I understand that anticipation in an inter partes review must be shown by a preponderance of the evidence. 5 Ex Page 10

11 C. Legal Standards for Obviousness 21. I understand that even if a patent is not anticipated, it is still invalid if the differences between the claimed subject matter and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person of ordinary skill in the pertinent art. 22. I understand that a person of ordinary skill in the art provides a reference point from which the prior art and claimed invention should be viewed. This reference point prevents one from using his or her own insight or hindsight in deciding whether a claim is obvious. 23. I also understand that an obviousness determination includes the consideration of various factors such as (1) the scope and content of the prior art, (2) the differences between the prior art and the asserted claims, (3) the level of ordinary skill in the pertinent art, and (4) the existence of secondary considerations such as commercial success, long-felt but unresolved needs, failure of others, etc. 24. I understand that an obviousness evaluation can be based on a combination of multiple prior art references. I understand that the prior art references themselves may provide a suggestion, motivation, or reason to combine, but other times the nexus linking two or more prior art references is simple common sense. I further understand that obviousness analysis recognizes that market demand, rather than scientific literature, often drives innovation, and that a 6 Ex Page 11

12 motivation to combine references may be supplied by the direction of the marketplace. 25. I understand that if a technique has been used to improve one device, and a person of ordinary skill in the art would recognize that it would improve similar devices in the same way, using the technique is obvious unless its actual application is beyond his or her skill. 26. I also understand that practical and common sense considerations should guide a proper obviousness analysis, because familiar items may have obvious uses beyond their primary purposes. I further understand that a person of ordinary skill in the art looking to overcome a problem will often be able to fit together the teachings of multiple publications. I understand that obviousness analysis therefore takes into account the inferences and creative steps that a person of ordinary skill in the art would employ under the circumstances. 27. I understand that a particular combination may be proven obvious merely by showing that it was obvious to try the combination. For example, when there is a design need or market pressure to solve a problem and there are a finite number of identified, predictable solutions, a person of ordinary skill has good reason to pursue the known options within his or her technical grasp because the 7 Ex Page 12

13 result is likely the product not of innovation but of ordinary skill and common sense. 28. The combination of familiar elements according to known methods is likely to be obvious when it does no more than yield predictable results. When a work is available in one field of endeavor, design incentives and other market forces can prompt variations of it, either in the same field or a different one. If a person of ordinary skill can implement a predictable variation, the patent claim is likely obvious. 29. It is further my understanding that a proper obviousness analysis focuses on what was known or obvious to a person of ordinary skill in the art, not just the patentee. Accordingly, I understand that any need or problem known in the field of endeavor at the time of invention and addressed by the patent can provide a reason for combining the elements in the manner claimed. 30. I understand that a claim can be obvious in light of a single reference, without the need to combine references, if the elements of the claim that are not found explicitly or inherently in the reference can be supplied by the common sense of one of skill in the art. 31. I understand that secondary indicia of non-obviousness may include (1) a long felt but unmet need in the prior art that was satisfied by the invention of 8 Ex Page 13

14 the patent; (2) commercial success of processes covered by the patent; (3) unexpected results achieved by the invention; (4) praise of the invention by others skilled in the art; (5) taking of licenses under the patent by others; (6) deliberate copying of the invention; (7) failure of others to find a solution to the long felt need; and (8) skepticism by experts. 32. I also understand that there must be a relationship between any such secondary considerations and the invention. I further understand that contemporaneous and independent invention by others is a secondary consideration supporting an obviousness determination. 33. In sum, my understanding is that prior art teachings are properly combined where a person of ordinary skill in the art having the understanding and knowledge reflected in the prior art and motivated by the general problem facing the inventor, would have been led to make the combination of elements recited in the claims. Under this analysis, the prior art references themselves, or any need or problem known in the field of endeavor at the time of the invention, can provide a reason for combining the elements of multiple prior art references in the claimed manner. 34. I understand that obviousness in an inter partes review must be shown by a preponderance of the evidence. 9 Ex Page 14

15 D. Legal Standards for Priority 35. I understand that, in order for a claim of a patent to be entitled to claim priority to an earlier filed provisional application, the earlier filed application must provide written description support for the claims of the patent application that claims priority to the earlier filed application. 36. I further understand that to satisfy the written description requirement, the applicant must convey with reasonable clarity to a person of ordinary skill in the art ( POSITA ) that, as of the filing date sought, the applicant was in possession of the invention. 37. I further understand that the determination of whether an applicant was in possession of an invention, as shown in the disclosure, requires an objective inquiry into the specification of the application. An applicant is in possession of the invention by describing the invention, with all its claimed limitations, rather than merely asserting that non-disclosed limitations make the invention obvious. III. OVERVIEW OF THE 488 PATENT A. Summary of the 488 Patent 38. The 488 patent is titled Method and Apparatus for Providing Tactile Sensations, and generally relates to outputting haptic effects based on either the amount of pressure applied to an input device, or a position of the input device. Ex. 1101, Abstract, 2:19-35, 8:58-9:4. An input device can be any one of a 10 Ex Page 15

16 number of devices, such as a tablet, mobile phone, gamepad, keyboard, computer mouse or other electronic devices, or alternatively can be mechanical input devices such as a mechanical switch. Id., 1:22-30; 3:8-17. For example, in certain embodiments, the 488 patent shows the input device as one with mechanicallyactuated switch buttons, (id., 9:60-65, Fig. 4), whereas in other embodiments, the input device is a touchscreen device, for example a PDA with a touchpad and a display. Id., 11:45-47, Fig Figures 3 and 4 below show an embodiment where the input device uses mechanically-actuated switch buttons. Figure 3 shows a mobile phone 14 with mechanically-actuated keys 10a through 10i. Ex. 1101, 8: Ex Page 16

17 40. As shown in further detail in Figure 4 below, keys 10a-10i are in communication with switches 19, which are further in communication with a controller (not shown). Ex. 1101, 8: As a user applies pressure to a key, the selected key will progress through a plurality of positions, including second position 25, third position 26, fourth position 27, and fifth position 28. Id., 8:58-9:1. The input device shown in Figure 4 includes actuator 61, which is configured to receive signals from a controller and output vibration to a metal diaphragm 20, causing PCB 62 to vibrate, then causing switches 19 to vibrate, and finally causing the selected key to vibrate. Id., 9: Ex Page 17

18 41. Figures 5 and 6 below disclose a PDA embodiment. The PDA 31 includes a pressure-sensitive touchpad 30, a display panel 33 (such as an LCD display), and a plurality of software-generated buttons or keys, called softkeys 36a-i, the configuration of which is shown in more detail in Figure 6 below. Ex. 1101, 11:3-13, 11: Ex Page 18

19 14 Ex Page 19

20 42. As shown, the display 33 is beneath touchpad 30, and is in communication with touchpad 30 through spacers 34, such that vibrations (output from actuator 64) can be received by touchpad 30 from display 33. Ex. 1101, 11: Softkeys 36b-i can represent multiple positions or multiple inputs, each input or position corresponding to a distinct amount of pressure applied to the softkey 36b-i. Id., 11: B. Claims Are Not Entitled to a Priority Date of November 1, The 488 patent is a continuation of the 260 patent (Ex. 1109), and claims priority to provisional application No. 60/335,493 (filed November 1, 2001) and provisional application No. 60/399,883 ( 883 application ). Based on legal principles, it is my opinion that at least claim 25 of the 488 patent is not entitled to the priority date of provisional application No. 60/335, The 883 application is a nine-page document that discloses a smartphone that uses mechanically-actuated keys, and also discloses touchpad embodiments that can include an LCD screen that displays buttons. Ex. 1115, 4, 10. The 883 application contains two figures that only show the embodiment of the smartphone with mechanically-actuated keys. Id., While the 883 patent discloses an integrated touchpad/display with buttons (or softkeys), it provides no disclosure on a position-based system as 15 Ex Page 20

21 arranged in the claim (i.e., a system that identifies a position of the input device and generates input and actuator signals associated with the position, based on a movement of a touchpad), and it does not disclose the particular claimed configuration of a processor as required by claim Thus, in my opinion, at least the following limitations lack written description: a processor configured to [perform limitations involving receiving and transmitting an input/actuator signal], the [first/second] input signal associated with the [first/second] position, the [first/second] actuator signal associated with the [first/second] position, and the display panel configured to receive the tactile sensations from the pressure-sensitive touchpad (claim 25); and a pressure calculator to calculate an amount of pressure based upon an amount of area of the interface object in contact with the display panel (claim 27). 47. More important than these aforementioned limitations, claim 25 as a whole is not disclosed. C. Person of Ordinary Skill in the Art 48. A person of ordinary skill in the art at the time of the alleged invention of the 488 patent (a POSITA ) would have had a Bachelors degree in computer science, electrical engineering, or a comparable field of study, plus approximately two to three years of professional experience with software engineering, haptics programming, human-computer interaction, or other relevant 16 Ex Page 21

22 industry experience. Additional graduate education could substitute for professional experience and significant experience in the field could substitute for formal education. D. Claim Construction 49. I understand from Apple counsel that in an inter partes review, claims are to be given their broadest reasonable interpretation ( BRI ) in view of the specification. 50. I understand that in ITC Investigation No. 337-TA-1004, the following documents may impact the scope of claim construction under the BRI: Immersion s Preliminary Proposed Claim Constructions (Ex. 1107), in which Immersion states how it believes certain terms of the 488 patent should be construed in that proceeding; the Chief ALJ s claim construction order that construes certain terms of the 488 patent (Ex. 1108); Immersion s technical domestic industry claim chart (Ex. 1105), in which Immersion shows how it believes that certain claims of the 488 patent encompass aspects of Immersion s technology; and 17 Ex Page 22

23 claim charts (Ex. 1104), in which Immersion shows how it believes that certain claims of the 488 patent allegedly encompass certain of Petitioner Apple s products. 51. I understand from Apple counsel that, for purposes of this proceeding, it is proper to request that Immersion be held to claim constructions that are as broad as those that Immersion has explicitly and implicitly publicly set forth in the documents mentioned above, and that I should assume that the explicit and implicit constructions in those claims charts are reasonable. I therefore have considered those documents in reaching my conclusions about what the claim terms mean under the BRI. 52. I understand that the standards used in the ITC and in a district court to interpret patent claims are different than those used by the PTO in this proceeding. I understand that the main difference is that in this proceeding, the claims are to be read as broad as is reasonable, based on the specification. I understand that this may cause the claims to cover certain things in this proceeding that a court might find are not within the scope of the claims in the ITC and district court proceedings. 53. In the following paragraphs, I provide a construction for certain claim terms based on their BRI in view of the specification, based on Immersion s 18 Ex Page 23

24 apparent belief about the scope of the claim terms, and based on the Chief ALJ s prior constructions in the ITC investigation. Claim Term pressure (claims 1, 9, 17, and 25-27) first/second haptic effect and first/second tactile sensation (claims 1, 9, 17, and 25) threshold (claims 1, 9, and 17) Immersion s Proposed Constructions application of force from a contact the first haptic effect can be the same as or different from the second haptic effect ; the first tactile sensation can be the same as or different from the second tactile sensation 1 level 1. No single interaction (claims 3-8, 11-16, 19-28) 54. In my opinion, the challenged claims do not require (expressly or by incorporation from independent claims 1, 9, 17, or 25) multiple haptic effects to be produced during a single interaction, i.e., a single, continuous contact by the user with the input device. I understand that in related IPR , the Board 1 The 488 patent is a continuation of U.S. Patent No. 7,336,260 ( 260 patent ) (Ex. 1109). The 260 patent includes independent claim 1 that requires a first, second, and third tactile sensations, and dependent claim 6 that requires a fourth tactile sensation different from the first tactile sensation. Ex at 20:33-46, 21:4-5. The doctrine of claim differentiation thus supports this construction under the BRI. 19 Ex Page 24

25 previously considered and rejected Patent Owner s claim construction that independent claims 1, 9, 17, and 29 require the multiple claimed haptic effects to be output during the single interaction. 55. Further, there is no single interaction limitation in any of the challenged claims for at least the following reasons. 56. First, none of the challenged claims (or any claim upon which a challenged claim may depend on) include any language that can reasonably be construed to require multiple pressures to be detected or multiple haptic effects to be provided during a single interaction. 57. Second, claims 1, 9 and 17 contemplate a single pressure and actuator signal during the claimed interaction with the graphical object, and nothing in the claims supports Immersion s assertion that the claims require detecting multiple pressure thresholds or providing multiple tactile sensations to the input device during a single touch interaction by the user. The conditional if statements in claims 1, 9 and 17 that result in one of two alternative haptic effects are expanded upon in dependent claims 8, 16 and 24 to result in one of four alternative haptic effects. The four alternative haptic effects recited in those dependent claims do not need to occur during a single interaction for the same reasons. 20 Ex Page 25

26 58. Third, the specification includes no description of determining whether multiple input/pressure signals received from an input device are generated in response to a single continuous user touch or separate touches, and it does not disclose basing the generation of a haptic effect on any such consideration. 59. Fourth, Immersion relies on a broader interpretation in the ITC proceedings, and thus the challenged claims should not be construed as limited to a single interaction. As shown at 19 of Exhibit 21 to Immersion s May 5, 2016 Complaint in the ITC (Ex. 1104), according to Immersion s theory of infringement, the Peek haptic effect occurs during a first interaction with a first graphical object (e.g., if the user presses the URL in the left figure hard enough), while the Pop haptic effect occurs during a second interaction with a second graphical object (e.g., if the user presses the map in the middle figure hard enough). See Ex ( stop pressing and you ll return to the screen where you found the address ). Thus, Immersion relies in the ITC on a broad reading of these claims that includes providing multiple effects during different interactions with different graphical objects. 60. Fifth, the prosecution history does not support any single interaction construction. Patent Owner may assert that the requirements to detect multiple 21 Ex Page 26

27 pressure threshold and provide multiple tactile sensations during a single touch interaction are compelled by the examiner s statement of reasons for allowance. This assertion is flawed for at least three reasons. First, the examiner s statement of reasons for allowance has nothing to do with a during a single touch interaction requirement. The examiner s statement must be understood in connection with the argument made by the applicants, who argued that Rosenberg discloses outputting haptic effects to a touch sensitive surface upon contacting a graphical object; however, Rosenberg does not disclose outputting haptic effects based on the sense amount of pressure when an object contacts the touch-sensitive surface. Ex. 1103, 16. Thus, the distinction drawn by the applicant and reflected in the examiner s statement was that the haptic effects were based on a detected pressure, not that multiple pressures were detected, or multiple tactile sensations were generated, during a single touch interaction. Second, unlike a statement made by an applicant, an applicant s silence in response to an examiner s reasons for allowance is not a valid basis for importing a limitation into the claims. Third, even a statement by the applicant cannot justify the importation of a limitation into a claim where, as here, there is no claim language to support the limitation. 22 Ex Page 27

28 2. cause a first haptic effect if the pressure is less than a first pressure threshold (claims 3-8, 11-16, 19-24) 61. This limitation should be construed broadly to require only that there be at least one pressure less than a first pressure threshold at which a first haptic effect is caused to occur. In particular, this limitation should not be construed in a manner that requires the first haptic effect to be caused in response to all pressures less than the first pressure threshold. Construing this limitation in a manner that requires the first haptic effect to be caused in response to all pressures less than the first pressure threshold would require causing a first haptic effect for even a zero pressure, or for a very small pressure. Nothing in the 488 patent indicates such a construction. Indeed, at least one of the preferred embodiments would be read out by such a requirement. Figure 4 below illustrates a pressure sensitive switch with a rest position 23 and multiple thresholds 25-28: 23 Ex Page 28

29 62. In this embodiment, no haptic effect is produced when the pressure is zero or greater than zero but less than the threshold 25. Ex.1101 at 15: Accordingly, the first haptic effect is not produced for all pressures below threshold 26, and construing this limitation as requiring that the first haptic effect be produced for all pressures including zero pressure and very low pressures would result in the exclusion of this embodiment from claims 1, 2, 9, 10, 17, and 18. A construction that excludes a [disclosed] embodiment from the scope of the claims is rarely, if ever, correct. Broadcom Corp. v. Emulex Corp., 732 F.3d 1325, 1333 (Fed. Cir. 2013) (quoting Accent Packaging, Inc. v. Leggett & Platt, Inc., 707 F.3d 1318, 1326 (Fed. Cir. 2013)). Thus, this limitation should be construed as 24 Ex Page 29

30 requiring only at least one pressure less than a first threshold at which a first haptic effect is caused to occur. 63. Additionally, Immersion relies on this interpretation in the ITC proceedings. As shown at 19 of Exhibit 21 to Immersion s May 5, 2016 Complaint in the ITC (Ex. 1104), according to Immersion s theory of infringement, the Peek haptic effect is the first haptic effect below the first threshold, but there is no haptic effect for lower forces that will only cause a Hint. 3. input device (claim 25) 64. Input device should be construed as a device by which a user can interact with an electronic device to provide instructions, responses, and other input to the electronic device, consistent with the Board s construction in IPR This term is not limited to the button embodiments and would include the disclosed mini-joystick and gamepad embodiments. 4. the input device moveable to the first position upon application of a first pressure to the input device, and moveable to the second position upon application of a second pressure to the input device, the second pressure greater than the first pressure (claims 25) 65. It is my opinion that this limitation in claim 25 should be construed broadly, consistent with Immersion s infringement allegations in the ITC and the description in the 488 patent, to encompass at least two types of devices: (1) a 25 Ex Page 30

31 touchscreen that is deflected in small, microscopic amounts upon the application of pressure to the touchscreen surface, and (2) an assembly of components, such as a mouse or a keyboard, that include a sub-component, such as a switch or button, that can be moved to first and second positions upon the application of first and second pressures. 66. With respect to the first type of device, as shown in Ex (claim chart illustrating Immersion s infringement contentions in the ITC investigation), Immersion alleges that this limitation is satisfied by a capacitive touchscreen that includes capacitive sensors [that] instantly measure microscopic changes in the distance between the cover glass and the backlight. Ex at 101. This is shown in the images below: Ex at 100 (emphasis mine); 26 Ex Page 31

32 Ex at 101 (emphasis mine). Thus, under the construction implied by Immersion s infringement allegations, it is sufficient (under this implied construction) that the cover glass of the touchpad be capable of moving, even by microscopic amounts, to a first position in response to a first pressure and to a different, second position in response to a second pressure. 5. standard 101-key keyboard 67. Standard 101-key keyboard (claims 7, 15, and 23) should be construed to mean the standard 101-key IBM keyboard. This term does not appear in the 488 patent specification, and a POSITA would understand this limitation to refer to the standard 101-key IBM keyboard because it became the standard for PC keyboards for many years. 27 Ex Page 32

33 IV. PRIOR ART A. JP Claims 3-6, 8, 11-14, 16, 19-22, and 24 are rendered obvious by Japanese Published Application No. H , which I will refer to as JP725. I understand from counsel for Apple that JP725 is prior art to the 488 patent. 69. JP725 is directed toward information displays and input devices, such as touch panels for factory automation equipment, automatic vending machines, automatic teller machines, home appliances and the like. Ex. 1111, [0001]. JP725 explains that one of the issues concerning the use of touch panels in devices like those discussed above is that a touch panel can leave a user unsure of whether the user s input was received. Id., [0003]-[0006]. 70. In order to address these and other problems, JP725 discloses a system with a touchpad in which feedback in the form of vibrations caused by piezoelectric vibrators or piezoelectric elements is provided to the user. Ex. 1111, [0014]. JP725 s information display system includes a control circuit CT, which JP725 discloses can be implemented by hardware or by software from memory using a microcomputer. Id., [0081], [0138]. Control circuit CT includes a position computing unit and operating force detecting unit, which detects a total force based on forces reported by individual piezoelectric elements. Id., [0082]-[0085]. Control circuit CT also includes an operating force determining unit, which inputs 28 Ex Page 33

34 the total force and four pressure thresholds, and then decides into which of five classifications (i.e., ranges) F0, F1, F2, F3, and F4 the total force belongs based on the four thresholds. Id., [0092]-[0097]. 71. When the total force belongs to ranges F1-F4, control circuit CT selects a drive mode for vibrating the touch panel according to the table shown in Figure 13 below, in which R 1 -R 6 represent regions of the touchscreen such as a button and R 0 represents no defined region (Ex. 1111, [0087]-[0091]): 29 Ex Page 34

35 72. Ex. 1111, [0107]-[0108]. The symbols S 11, S 12 in the table represent any of the various drive modes such as those shown in Figure 14 below, which can vary by amplitude, frequency and length of time. Ex. 1111, [0109]-[0110]. 73. As demonstrated below, this and other information in JP725, combined with the knowledge and ordinary skill of a POSITA, renders claims 3-6, 8, 11-14, 16, 19-22, and 24 obvious. 1. Limitation 9.pre: A system, comprising: 74. In my opinion, JP725 renders obvious a system. 75. JP725 renders obvious a system. JP725 discloses an information display device that is shown in various embodiments, each of which is a system. Ex. 1111, Abstract, [0001]. JP725 s primary embodiments include an ATM 30 Ex Page 35

36 embodiment having a touchscreen, (Ex. 1111, Figs and accompanying text, [0040]-[0041]) (ATM embodiment), and a hand-held embodiment having a touchscreen (Ex. 1111, Figs and accompanying text) (PDA embodiment). 76. JP725 discloses two primary variations of a touchscreen with corresponding modifications to control circuit CT. Ex. 1111, Figs. 7 and 16 and accompanying text. These variations are explained in the context of the ATM embodiment, but are incorporated by reference to the PDA embodiment. Ex. 1111, [0146], [0159]. As will be explained in further detail below, in both variations the control circuit CT receives the input signals from the touchscreen, outputs display signals to the touchscreen, performs processing to detect interactions with graphical objects, and generates and outputs the actuator signal to the actuators. 77. JP725 expressly discloses that each of its disclosed embodiments, while directed to various applications, is a variation of the same basic embodiment. For example, the ATM embodiment is disclosed at Figures 1-16, but JP725 discloses that everything is the same between the ATM variations that use the different touch screen and control circuit CT. Ex. 1111, [0133], [0137]-[0138], [0143]-[0144]. Similarly, JP725 discloses that everything in the PDA embodiment (disclosed at Figures 17-19) is the same as the ATM embodiments, with the only differences expressly identified, meaning the PDA can also use both variations of 31 Ex Page 36

37 touchscreen and control circuit CT. Ex. 1111, [0146]. Other potential modifications are disclosed in Figure 23 and text from [0160] to [0177], and JP725 expressly discloses that those modifications are applicable to all other embodiments. Ex. 1111, [0166]. 78. Thus, a POSITA would understand these express disclosures as confirmation that the teachings of JP725 can be taken, as a whole, as an anticipatory reference in the analysis below that is directed to the individual claim elements. Alternatively, combining elements from all disclosed embodiments and modifications in JP725 would have been obvious to a POSITA, as JP725 expressly teaches that these embodiments and modifications are design choices applicable to all disclosed embodiments with disclosed advantages and disadvantages. 2. Limitation 9.a: a touch-sensitive input device configured to output a sensor signal indicating an object contacting the touch-sensitive input device and a pressure of the contact 79. JP725 renders this limitation obvious. 80. In the ATM embodiment, the ATM comprises a touchscreen that includes a transparent touch panel 10 over a display panel 20. Ex. 1111, Figs. 3, 4, [0043]-[0044]. The touch panel 10 is mounted on four piezoelectric elements E1- E4. Id., Figs. 3, 4, [0046]. Figures 3 and 4 below show this configuration. 32 Ex Page 37

38 81. The ATM embodiment is described using the first variation of touchscreen and control circuit CT. Piezoelectric elements E1-E4 detect contact and provide haptic feedback to the touch panel 10. Ex. 1111, [0048]. Piezoelectric 33 Ex Page 38

39 elements E1-E4 output sensor signals e1-e4 to an operation unit 51 as shown in Figure 7 below. Ex. 1111, Fig. 7, [0082]. Operation unit 51 uses these sensor signals from each of E1-E4 to calculate the position of the contact (signal SP) and the force of the contact (signal SF). Ex. 1111, Fig. 8, [0083]. JP725 s calculated force, represented as signal SF, is a pressure of the contact under the BRI of that term. As will be explained in further detail, signal SP is used to generate signal SR, and signal SF is used to generate signal FB, which are input to unit 72. Ex. 1111, Figs. 7, As discused in JP725, components such as the operation unit 51 and force determining unit 54 can be implemented in either hardware circuits or in software by a microcomputer. Ex. 1111, [0081]. In embodiments in which the operation unit 51 and force determining unit 54 are implemented in hardware circuits, a POSITA would understand (or at least find obvious) that such circuits can be part of the touchscreen. In particular, a POSITA would have found it obvious to include all of the components inside the dashed lines shown in the annotated version of Figure 7 below as part of the touchscreen because those components perform relatively low-level functions that are easily implemented in hardware and, with respect to unit 75, involves an analog signal. 34 Ex Page 39

40 83. In contrast, the components of Figure 7 outside the dashed lines, such as the informationn processing unit 60, perform functions described in JP725 (e. g., in paragraphs [0106]-[0107]) that more readily lend themselves to implementation in a microprocessor. Alternatively, a POSITA would have found it obvious to implement all of the components of Figure 7, all of the components other than the operation unit 51 and the drive unit 75, or only the touch/operating panel and display, as part of the touchscreen. A POSITA would understand that which components to integrate into a processor (which need not be a single chip) is a design choice for each implementation. The level of ordinary skill in the art is reflected by the 488 patent which, like JP725, also primarily describes the functionality in terms of a controller, and then makes clear that whether and how 35 Ex Page 40

41 the controller is implemented into a microprocessor, DSP, ASIC, or state machine is well within the skill of a POSITA. Ex at 4:67-5:6. The specific implementation chosen by a POSITA in any particular instance will depend on many factors, including cost and speed of the microprocessor. Nonetheless, the various alternatives discussed above are all obvious to a POSITA. 84. The ATM embodiment uses a second variation of touchscreen and control circuit CT, shown in Figures and described at [0133] [0144]. In this variation, the touch panel 10T is a resistive type touch panel that uses transparent electrodes on the display surface to perform the function of sensing contact and providing position information. Ex. 1111, Figs. 15, 16, [0133]-[0134]. Resistive touch panels provide position information directly, meaning signals are output by this touch panel to operating position specification 51T (and its output of position signal SP), and are thus an additional disclosure of limitation [9.a]. Ex. 1111, Fig. 16, [0134]. 85. JP725 further discloses that the touch panel 10T may instead be an optical, acoustic, or capacitive touch panel, each of which were well-known types of touch panels. Ex. 1111, [0135]. In this variation, the four piezoelectric elements E1-E4 are used to provide force (pressure) information to operation unit 51F, which outputs force signal SF. Ex. 1111, Fig. 16, [0141]. JP725 notes that 36 Ex Page 41

42 most of the elements of the control circuit CT in FIG. 16 have the same configuration and function as an FIG. 7. Ex. 1111, [0138]. The boundary of the touch-sensitive input device (for the second variation) can be divided in a manner similar to annotated Figure 7 above, except that instead of operating unit 51, the device includes operation unit 51F and operating position specification 51T. Ex. 1111, Fig The PDA embodiment can use the same two variations of touchscreens without further modification to output a sensor signal and, therefore, disclose this limitation in the same manner as the other embodiments. Ex. 1111, [0145], [0159]. 3. Limitation 9.b: an actuator coupled to the touch-sensitive input device, the actuator configured to receive an actuator signal and output a haptic effect to the touch-sensitive surface based at least in part on the actuator signal, wherein the actuator signal is configured to cause a first haptic effect if the pressure is less than a first pressure threshold and cause a second haptic effect if the pressure is between the first pressure threshold and a second pressure threshold 87. JP725 renders obvious this limitation, an actuator coupled to the touch-sensitive input device, the actuator configured to receive an actuator signal and output a haptic effect to the touch-sensitive surface based at least in part on the actuator signal, wherein the actuator signal is configured to cause a first haptic effect if the pressure is less than a first pressure threshold and cause a second 37 Ex Page 42

43 haptic effect if the pressure is between the first pressure threshold and a second pressure threshold. 88. For the first part of this limitation (an actuator coupled to the touchsensitive input device), JP725 discloses piezoelectric actuators E1-E4 (elements 30 in Figures 3 and 4 above), which are also pressure sensors, that are coupled to the touch panel. Ex. 1111, [0046]-[0049]; see also limitation [9.a]. The actuators are the same for all disclosed embodiments, and regardless of which variation of control circuit CT is used to generate and provide the actuator signal. See Ex. 1111, Figs. 7, 16, 21 and accompanying text. 89. The piezoelectric actuators are configured to receive an actuator signal from the drive unit 75 portion of control circuit CT. Ex. 1111, Fig. 7; Fig. 16. JP725 describes how the actuator signal causes the piezoelectric elements E1- E4 to generate a haptic effect, such as a vibration. Ex. 1111, Abstract. JP725 also discloses that other haptic effects may be provided, such as pulsed displacement. Ex. 1111, [0122]. Further examples of actuator signals are depicted in Figure 14, and include continuous vibrations of selectable amplitudes and frequencies, as well as short bursts of vibrations and discrete pulses. Ex. 1111, Fig. 14, [0110]. JP725 emphasizes that [t]his is felt by the user because it causes the entire touch panel to 38 Ex Page 43

44 vibrate. Ex. 1111, [0123]. The haptic output notif[ies] the operator that the operation was received.... Ex. 1111, [0113]. 90. JP725 further renders obvious the wherein clause of this limitation. JP725 discloses detecting forces (pressures) that fall into one of five classifications (i.e., ranges), F0, F1, F2, F3, and F4, based on four pressure thresholds Fh1, Fh2, Fh3, Fh4. Ex. 1111, [0092]-[0095]. A POSITA would understand that the pressure thresholds Fh1 through Fh4 are each a pressure threshold under the BRI for that term. 91. Pressure range F1 is disclosed in JP725 as a pressure greater than or equal to threshold Fh1 and less than threshold Fh2. Ex. 1111, [0096]; Fig. 11. When the user presses the touchscreen in area R 1 at a pressure in the F1 range, a first haptic effect S 11 is produced, as shown in Figure 13 below. JP725 thus discloses a pressure (in the F1 range) below a first threshold (threshold Fh2) for which JP725 s system causes a first haptic effect (the S 11 haptic effect). Thus, JP725 discloses causing a first haptic effect (S 11 ) for a pressure (in the F1 range) less than a first pressure threshold (threshold Fh2). 39 Ex Page 44

45 92. For the limitation cause a second haptic effect if the pressure is between the first pressure threshold and a second pressure threshold, JP725 discloses that pressure range F2 represents pressures between Fh2 and Fh3, as shown in Figure 11 below. Ex. 1111, [0096], Fig Ex Page 45

46 93. As shown in Figure 13, when the user presses the touchscreen in an active area R 1 at a pressure in the F2 range, a second haptic effect S 12 is produced. Thus, JP725 discloses causing a second haptic effect (S 12 ) for a pressure (in the F2 range) between the first pressure threshold (threshold Fh2) and a second pressure threshold (Fh3). 4. Limitation 9.c: a processor in communication with the sensor and the actuator, the processor configured to 94. JP725 renders obvious a processor in communication with the sensor and the actuator in all disclosed embodiments. 95. As discussed above regarding preamble [9.pre] and limitations [9.a] (sensor) and [9.b] (actuator), the ATM and PDA embodiments use the same primary variations of control circuit CT. As further established with respect to 41 Ex Page 46

47 limitation [9.a], JP725 teaches to a POSITA that t any and up to all of the functional units disclosed for controllerr CT can be implemented as discrete hardware components or as a microprocessor running software from memory. 96. In the first variation, JP725 discloses that the processor (comprising the components outside the dashed line in annotated Figure 7 below) is in communication with the sensor and actuator through drive mode selecting unit 72 which receives signals SR (location) and FB (pressure) from region determining unit 52 and operating force determinin ng unit 54. Ex. 1111,, Figs. 7, 16, [0108]. The second variation discloses this limitation forr the same reason (i.e., the components outside the dashed line are similar, if not identical, as shown in Figure 16). 42 Ex Page 47

48 97. JP725 also discloses other possible configurations for the first variation. JP725 expressly discloses that control circuit CT can be implemented as a processor (MPU) executing software from memory. Next, the configuration and operation of the control circuit unit CT (FIG. 7) of the information display device 100 will be described based on the principles described above. Note that while an example where the control circuit unit CT is configured using a hardware circuit is illustrated here, this circuit can also be realized with software using a microcomputer. In this case, the following circuit portions are functionally realized using the MPU and memory of said microcomputer. 98. Ex. 1111, [0081] (emphasis mine). In the first variation, [t]he following circuit portions referred to by JP725 in defining the control circuit CT can comprise blocks 51-57, 60-62, in Figure 7. In that case, the processor (MPU) is directly in communication via signal lines with the piezoelectric sensor/actuators. Additionally, a POSITA would understand that control circuit CT depicted in Figure 7 performs the functions of a processor, and could be implemented in hardware, or with software executed by a microcontroller. However, under the BRI, the claims do not require the processor be directly in communication with the other components or directly transmit or receive signals with them. 99. In the second variation, the sensor comprises the same piezoelectric elements as the first variation for sensing force, and a resistive, capacitive, optical, 43 Ex Page 48

49 or acoustic touch panel for sensing contact location. The functions performed by operation unit 51 in the first variation are now divided between operating position specification 51T and operation unit 51F to accommodate the different touchscreen hardware, as shown in Figure JP725 expressly discloses that the second variation of control circuit CT in Figure 16, like the first variation in Figure 7, can be implemented as a processor executing software in memory: FIG. 16 is a configuration diagram of a control circuit CT using the display operating unit DP in FIG. 15, and while this circuit is described as a hardware circuit, as was the circuit in FIG. 7, the functions of these circuits can be expressed using software. Ex. 1111, [0138] (emphasis mine). In that case, blocks 51F/T, 52-57, 60-62, and in Figure 16 can comprise the processor running software from memory, and would be in communication via signal lines with piezoelectric sensor/actuators, and resistive, capacitive, optical, or acoustic touch panel. Ex. 1111, Fig. 16 and accompanying text. Additionally, a POSITA would understand that control circuit CT shown in Figure 16 performs the functions of a processor, and could be implemented in hardware, or with software executed by a microcontroller Further, JP725 expressly describes the signal paths between the portion of the processor referred to as processing unit 60. Ex. 1111, [0106] (processor 60 receives sensor signal SR indicating the selection of a graphical 44 Ex Page 49

50 object on input sensor); [0125] (processing unit 60 provides actuator signal to stop actuator). 5. Limitation 9.d: [the processor configured to] output a display signal configured to display a graphical object on the touch-sensitive input device 102. JP725 renders obvious this limitation. JP725 discloses that the processor (in all variations) is configured to output a display signal to the touch screen comprising display 20 and touch panel 10. Ex. 1111, Fig. 7 (showing processing unit 60 in communication with display driver 71 and LCD panel 20), Fig. 16 (same), Fig. 22 (same). As further established with respect to limitation [9.a], JP725 teaches to a POSITA that any and up to all of the functional units disclosed for controller CT can be implemented as discrete hardware components or as a microprocessor running software from memory In the ATM embodiment, the processor is configured to output a display signal configured to display a graphical object (ATM menu comprising graphical objects R 1 to R 7 ) on the touch-sensitive input device (touchscreen comprising touch panel 10 and display 20) as shown in Figure 4 below. This includes for example the R 1 Deposit button, the R 2 Payment button, and the R 3 Withdrawal button. Ex. 1111, Fig. 4, [0047] (deposit), [0106] (withdrawal). 45 Ex Page 50

51 104. In the PDA embodiment, as shown in Figures below, JP725 discloses that the processor is configured to output a display signal displaying graphical objects R 1 -R 4 on the touchscreen. Ex. 1111, Figs. 17, 18, [0145]. 46 Ex Page 51

52 6. Limitation 9.e: [the processor configured to] receive the sensor signal from the touch-sensitive input device, 105. JP725 renders obvious this limitation. JP725 discloses that the processor receives the sensor signal from the touch-sensitive input device (touchscreen), as established above with respect to claim elements [9.a] and [9.c]. Specifically, the processor in the first variation receives the sensor signal from the piezoelectric elements E1-E4, which it uses to determine both position and force of contact. Ex. 1111, [0048], [0082], [0083]. This is shown in Figure 7 below, annotated to show portions of the touch-sensitive input device and the processor. As further established with respect to limitation [9.a], JP725 teaches to a POSITA that any and up to all of the functional units disclosed for controller CT can be 47 Ex Page 52

53 implemented as discrete hardware components or as a microprocessor running software from memory In the second variation, the processor receives the sensor signals from the touchscreen, specifically the piezoelectric elements E1-E4 (whichh it uses to determine the force of contact) and the resistive or other touch panel types (which it uses to determine the position of the contact). Ex. 1111, [0133]-[0135], [0141], Fig Ex Page 53

54 7. Limitation 9.f: [the processor configured to] determine an interaction between the object contacting the touchsensitive surface and the graphical object based at least in part on the pressure 107. JP725 discloses logic product circuit 57 shown in Figure 7 (reproduced above for limitation [9.d]) and discussed in paragraph [0104]-[0105] that a POSITA would have found obvious to implement using a processor. Doing so would have been obvious for the reasons discussed above for limitation [9.c] for both the first and second variations. Furthermore, it should be understood that, although circuit 57 is shown in Figure 7 as a simple two-line AND gate, JP725 discloses that the actual functionality can be more complicated. For example, JP725 expressly discloses that the SR signal can be a multi-bit signal in order to distinguish between the multiple areas and the non-activity level. Ex. 1111, [0091]. The multi-bit signal would need to be further processed in order to be used as an input to a simple AND gate as shown in Figure 7. As discussed above for limitation [9.c], while doing so in a dedicated hardware circuit is possible, a POSITA would have found it obvious to implement this processing via a processor rather than in discrete hardware. Thus, JP725 renders obvious the implementation of the functionality of circuit 57 in a processor. Id.,. As further established with respect to limitation [9.a], JP725 teaches to a POSITA that any and up to all 49 Ex Page 54

55 of the functional units disclosed for controller CT can be implemented as discrete hardware components or as a microprocessor running software from memory JP725 renders obvious the remainder of this limitation. In the ATM embodiment, the processor determines an interaction (selecting an ATM button R 1 - R 7 ) between the object (a user s finger) contacting the touch-sensitive surface (the ATM touchscreen) and the graphical object (ATM buttons R 1 -R 7 ). Ex. 1111, Fig. 4, [0047]. In this particular example, the nature of the interaction determined is that of relative position between the contacting object and that of the displayed graphical object R 1 -R 7, and the force with which the user presses the button. Both interactions are types of interactions disclosed in the 488 patent. Ex. 1101, Fig. 8 and accompanying text. Figure 4 below discloses the ATM menu with ATM buttons R 1 -R 7, including Deposit (R 1 ), Payment (R 2 ), and Withdrawal (R 3 ) buttons. Ex. 1111, Fig Ex Page 55

56 109. Ex. 1111, Fig. 4. JP725 also discloses in the ATM embodiment that, after the user presses the Withdrawal (R 3 ) button (causing the display of a numeric keypad on the ATM screen), the processor displays a numeric keypad and is configured to detect an interaction (selecting a numeric button) between the object (the user s finger) contacting the touch-sensitive surface (ATM touchscreen) and the graphical object (ATM numerical button) based in part on pressure (force applied to the button and touchscreen). Ex. 1111, [0120] In the PDA embodiment, JP725 discloses that the processor determines an interaction (selection of buttons) between the object (the user s thumbs) contacting the touch-sensitive surface (PDA touchscreen) and the 51 Ex Page 56

57 graphical object (buttons R1-R4) based at least in part on pressure (user applying force to the button and touchscreen). Ex. 1111, Figs. 17, 18, [0040], [0146]. Figures 17 and 18 below show the PDA buttons (including R 1 Return, R 2 Enter, R 3 Down, R 4 Up) JP725 discloses significant implementation details regarding how the processor is configured to make the above determinations based on the position and force of the contact much more detail than the 488 patent itself. Both JP725 and the 488 patent disclose touchscreens comprised of two layers a display underneath a transparent touchpad. Ex. 1111, Fig. 3; Ex. 1101, Fig. 6. A POSITA would understand that the graphical object is displayed by output signals sent to the display portion, and contacting the displayed button results in input signals 52 Ex Page 57

58 from the touchpad portion. The processor then determines whether the input signals correlate sufficiently with the displayed graphical objects to infer a selection of a button or other intended functionality of the user interface device In JP725, the processor receives a signal from the operating surface 11 indicating the position of a contact. Ex. 1111, Abstract. In this first variation, the input signal is from the piezoelectric elements E1-E4. The calculations presented at paragraphs [0049]-[0080] of JP725 provide the XY coordinates of the point of contact P at which an object (such as a finger) contacts the display surface, which is represented by a signal SP output by the operation unit 51, shown in Figure 7 and described at [0086] In the second variation, the input position signal is from the resistive, capacitive, optical or acoustic touch panel. Ex. 1111, Fig. 16, [0133], [0137]. This signal is received by the processor as shown in Figure 16 and as explained for limitations [9.a] and [9.c]. Ex. 1111, [0134], [0141] Regardless of whether SP is generated from operation unit 51 in the first variation, or operating position specification 51T in the second variation, the next step in determining a positional interaction between the object contacting the touch-sensitive surface and the graphical object being displayed is determining whether the contact location represented by operating position signal SP is inside 53 Ex Page 58

59 of a region corresponding to a displayed graphical object. This is done through determining unit 52, which then outputs signal SR to unit 72 where a determination is made as to what actuator signal should be output, based upon a force classification and a location of an object contacting the touch-sensitive input device JP725 further discloses that, in each of the ATM and PDA embodiments, the processor also determines an interaction (force level) between the object contacting the touch sensitive surface and the graphical object. When an object (such as the user s finger) contacts the touch-sensitive surface at an operating region R corresponding to a graphical object as described above, the MPU running software (illustrated as control circuit CT) additionally determines and classifies the level of force being applied to the region of the displayed graphical object. Ex. 1111, [0092]-[0104] ( Determination of the Operating Force ) In both variations, JP725 describes this functionality as being implemented by operating force determining 54 and operating force classification 55, in conjunction with force thresholds for graphical object provided by processing unit 60 and unit 72. Ex. 1111, Figs. 11, 12, [0051], [0086], [0092]- [0104]. Thus, for example, if the operating force signal SF indicates a magnitude 54 Ex Page 59

60 that is greater than threshold Fh1 and less than threshold Fh2, then the operating force determining signal FB would indicate a force classification of F1. Ex. 1111, [0095]-[0097]; Fig. 11. These thresholds and determinations are based on which graphical objects operating region was contacted. Ex. 1111, [0093] To the extent Patent Owner contends that the claimed interaction is limited to a single interaction, JP725 still renders obvious this limitation (and related limitations in claims 1, 17, and 25). This is shown in the example when a user selects the Withdrawal button and is able to access the interface where the user can specify an amount to withdraw. A POSITA would understand that a user, desiring to withdraw $100 for example, could enter a 1, followed by a 0, followed by another 0. After the user has input the numbers 1 and 0, the user can maintain contact with the portion of the touch panel that corresponds to the button displaying the number 0, and while applying less pressure (i.e., a pressure below the pre-determined threshold), the user can then apply sufficient pressure (i.e., a pressure exceeding the pre-determined threshold) that causes the device to recognize an input of 0 and output a corresponding haptic effect. In this example, the user never takes its finger off the number 0 button displayed on the display of the information display device. 55 Ex Page 60

61 8. Limitation 9.g: [the processor configured to] generate an actuator signal based at least in part on the interaction 118. For each of the interactions in each embodiment, JP725 renders obvious a processor configured to generate the actuator signal based at least in part on the interaction. JP725 discloses that the processor (control circuit CT) includes a drive mode selecting unit 72 with table 72a. Ex. 1111, Figs. 7, 13, 16, 22 and accompanying text. As further established with respect to limitation [9.a], JP725 teaches to a POSITA that any and up to all of the functional units disclosed for controller CT can be implemented as discrete hardware components or as a microprocessor running software from memory In the ATM and PDA embodiments, JP725 discloses that the processor will generate the actuator signal (V) based at least in part on the interaction (signal SR, representing the selection of ATM button R 1 -R 7 or Region R 0, and force FB representing the force applied) and haptic effect data (S 01 to S nn ) in table 72a. Ex. 1111, Fig. 4 (ATM menu), Fig. 7 (drive signal V), Fig. 13 (lookup table 72a) For example, JP725 discloses determining the selection of the Deposit button based on the contact location and force and providing the corresponding haptic effect based on that selection. Ex. 1111, [ ]. Similarly, in the PDA embodiment, JP725 discloses that the processor will 56 Ex Page 61

62 generate the actuator signal (V) based at least in part on the interaction (signal SR, representing the selection of PDA button R 1 -R 4 and force FB representing the force applied) and haptic effect data (S 01 to S nn in table 72a). Ex. 1111, Fig. 7 (drive signal V), Fig. 13 (table 72a), Figs (buttons R 1 -R 4 ), [0146] (referencing same features as ATM embodiments) JP725 provides additional disclosure of how the drive mode that produces the actuator signal is selected based on position and force-level interactions at paragraphs [0108] [0126] under a section entitled, Drive Mode Selection. JP725 explains that the drive mode defines the haptic effect that will be generated, and that this drive mode is based on the classifications of the inoperation region and the operating force, i.e. based on the determined interactions. Ex. 1111, [0108]. 9. Limitation 9.h: [the processor configured to] transmit the actuator signal to the actuator 122. JP725 renders obvious this limitation. JP725 discloses that a processor transmits the actuator signal (drive mode parameter signal V) to the piezoelectric actuators via drive unit 75. Ex. 1111, Abstract, Figs. 7, 16, [0115], [0122] ( drive mode parameter signal V output form the drive mode selecting unit 72 is applied to the piezoelectric element drive unit 75 ), [0123], claims 1-3, 15 ( drive signals ). As further established with respect to limitation [9.a], JP Ex Page 62

63 teaches to a POSITA that any and up to all of the functional units disclosed for controller CT can be implemented as discrete hardware components or as a microprocessor running software from memory. 10. Claim 11: The system of claim 9, wherein the processor is configured to output the actuator signal when the object contacts the touch-sensitive device at a location not corresponding to the graphical object JP725 s ATM embodiment (shown below in Figure 4) discloses several buttons (R 1 -R 7 ) and an area outside those buttons (region R 0 ). When a user contacts region R 0 using a finger, a weak vibration will be generated. Ex. 1111, [0112]. Thus, this haptic effect is generated when the user is contacting the touchscreen at a location not corresponding to the graphical object. 58 Ex Page 63

64 124. Alternatively, in the ATM embodiment, the user can instead select the Deposit button (instead of the Withdrawal button described above). In that scenario, the actuator signal is generated when the user s finger contacts the touchscreen at a location not corresponding to the Withdrawal button. 11. Claim 12: The system of claim 9, wherein the display signal is configured to display a keypad comprising a plurality of softkeys JP725 s ATM embodiment discloses that the display signal of the ATM is configured to display a keypad comprising a plurality of softkeys (software-generated buttons R1 through R7), as shown in Figure 4 below This is consistent with the 488 patent specification, which describes (in the context of Figure 5) that the display can include a plurality of software- 59 Ex Page 64

65 generated buttons or keys, called softkeys 36 a-i. Ex. 1101, 11: Additionally, once the user selects a softkey, such as Withdrawal (button R 3 ), the display changes to a withdrawal amount input screen, which simulates a numeric keypad, and for each key pressed, a one shot displacement like that in FIG. 14(f) may be applied each time the keypad is pressed. Ex. 1111, [0120] Further, a POSITA would understand that displaying a numeric keypad for use on the disclosed ATM touchscreen would comprise displaying a plurality of keys comprising one softkey for each digit from 0 to 9, such that a user could provide numerical entries for inputting the dollar amount, for example, of a withdrawal or deposit. 12. Claim 13: The system of claim 12, wherein the haptic effect is caused to be output when a user contacts the touchsensitive device at a location corresponding to a softkey in a home position Each of JP725 s disclosed softkeys (with respect to claims 9 and 12) are disclosed to be in a home position in that each is disclosed to be in its initial position. Ex. 1111, Figs. 4, 17, 20, 21. The term home position is not used in the specification of the 488 patent. A POSITA would consider the home position to be the original configuration or position of the softkeys. Nothing in claim 13 requires that the softkey be moveable to a second position that is different from its home position. And even if it did, JP725 discloses that the softkeys can change in 60 Ex Page 65

66 color [0107], arrangement [0120] and position [0155] based on the user s interactions, as more fully explained for limitations [9.d] and [9.f]. The 488 patent discloses that a home button can have a distinct tactile sensation. JP725 discloses that the softkeys have distinct tactile sensations. Fig. 13 (showing different sensations for different regions). 13. Claim 14: The method of claim 12, wherein the plurality of softkeys comprises one softkey for each digit from 0 to JP725 discloses that the plurality of softkeys comprises a numeric keypad for performing a Withdrawal at an ATM. It would have been obvious to a POSITA to implement such a numeric keypad with one softkey for each digit from 0 to 9 so that the user could input the amount of money to withdraw. This same numeric keypad would be required for other displayed menu functions in Figure 4, such as Money Transfer, Payment via Bank Deposit Transfer, and Passbook Entry JP725 also discloses applications using cellular phones. Ex. 1111, [0006]. In a cellphone (or PDA) with a touchscreen, it would have been obvious to a POSITA to implement a numeric keypad with a plurality of softkeys comprising one softkey for each digit from 0 to 9, so that the user could make a phone call. Moreover, during the prosecution of the 488 patent, the Examiner found this limitation inherent in prior art cell phones with touchscreens. Ex. 1103, Ex Page 66

67 14. Limitation 16.a: The method of claim 9, wherein the graphical object comprises a first graphical object and a second graphical object, and 131. As discussed above, JP725 s ATM embodiment discloses that the display signal of the ATM is configured to display buttons (corresponding to regions R 1 -R 7 ). In that embodiment, the Withdrawal button (R 3 ) can be the first graphical object, and the Deposit button can be the second graphical object, as shown below in Figure 4. JP725 discloses that other combinations of buttons (i.e., graphical objects) are possible. See, e.g., Ex. 1111, [0047] ( a variety of information can be displayed on the liquid crystal display panel 20 ). 62 Ex Page 67

68 15. Limitation 16.b: if the object contacts the first graphical object, the first haptic effect is configured to be output if the pressure is less than the first pressure threshold and the second haptic effect is configured to be output if the pressure is between the first pressure threshold and the second pressure threshold, and 132. See limitations [9.b] and [9.f]. 16. Limitation 16.c: if the object contacts the second graphical object a third haptic effect is configured to be output if the pressure is less than the first pressure threshold and a fourth haptic effect is configured to be output if the pressure is between the first pressure threshold and the second pressure threshold JP725 s ATM embodiment discloses multiple buttons, including a Deposit button. For the same reasons discussed above for limitations [9.a] and [9.f] through [9.h], when the user selects the Deposit button (the second graphical object), then depending upon whether the detected pressure is below the first pressure threshold (e.g., Fh2), or between the first pressure threshold and the second pressure threshold (e.g., Fh2 and Fh3), the actuator will output an actuator signal corresponding to signal S 11 or S 12, as shown in Figure 13 below. In contrast, if the Withdrawal button is selected (the first graphical object), then an actuator signal correspond to signal S 31 or S 32 (assuming signals in Figure 13 are labeled in a manner consistent with R 1, R 2, R 6, and R 0 ) will be output. 63 Ex Page 68

69 17. Limitation 1.pre: A method, comprising: 134. As established above for limitations [9.pre] and [9.c], JP725 discloses a system for performing the claimed functions, which correspond to the steps claimed in this method. Accordingly, JP725 discloses or renders obvious the method of using that system The table below represents a mapping of the limitations of claims 1, 3-6, and 8 to claims 9, 11-14, and 16. Claim / Claim Limitation JP725 [1.a] [1.b] [1.c] See limitations [9.a], [9.c], and [9.d]. See limitations [9.a], [9.c], and [9.e]. See limitations [9.c] and [9.f]. 64 Ex Page 69

70 [1.d] [1.e] See limitations [9.b], [9.c], and [9.g]. See limitation [9.h]. Claim 3 See claim 11. Claim 4 See claim 12. Claim 5 See claim 13. Claim 6 See claim 14. Claim 8 See claim Limitation 17.pre: A computer readable-medium comprising program code, comprising 136. JP725 discloses that the components shown in Figures 7 and 16 can be implemented by hardware circuits or by software using a microcomputer. Ex. 1111, [0081], [0138]. Specifically, JP725 discloses that when the components of Figure 7 are realized using software and a microcomputer, they are functionally realized using the MPU and memory of said microcomputer, and similarly the functions of [the control circuit CT shown in Figure 16] can be expressed using software. Id A POSITA would have understood that MPU was an abbreviation for a microprocessor unit. A POSITA would further have found it obvious that the disclosed software for realizing the functions of the circuits described therein would be stored in the microcomputer memory referred to in the same paragraph, 65 Ex Page 70

71 as this was a common arrangement that was well known in the art at the time of the alleged invention. Thus, JP725 discloses that it is possible to implement the control circuit components of Figures 7 and 16 using a MPU controlled by software (program code) stored in a microcomputer memory (computer readablemedium) The table below represents a mapping of claims 17, 19-22, and 24 to claims 9, 11-14, and 16. Claim / Claim Limitation JP725 [17.a] [17.b] [17.c] [17.d] [17.e] See limitations [9.a], [9.c], and [9.d]. See limitations [9.a], [9.c], and [9.e]. See limitations [9.c] and [9.f]. See limitations [9.b], [9.c], and [9.g]. See limitations [9.a], [9.c], and [9.d]. Claim 19 See claim 11. Claim 20 See claim 12. Claim 21 See claim 13. Claim 22 See claim 14. Claim 24 See claim Ex Page 71

72 B. Tsuji and Roysden 139. U.S. Pat. No. 5,575,576 ( Roysden ) issued on November 19, 1996 and is prior art to the 488 patent under 35 U.S.C. 102(b). In my opinion, Tsuji in view of Roysden renders obvious claims 7, 15, and Claim 7, 15 and 23 recite that the plurality of softkeys comprises the key configuration of a standard 101-key keyboard. In my opinion, this limitation is rendered obvious for the reasons set forth below JP725 discloses various applications for the information display device, including ATM and PDA embodiments. It would have been obvious to a POSITA to implement the user interface to resemble the standard 101-key keyboard in applications where alphanumeric characters (and non-alphanumeric characters) are entered by a user. It would have been obvious to use the standard 101-key keyboard configuration, introduced by IBM in 1986, because that was the standard for the PC industry for many years and users were familiar with the layout of this configuration As taught by Roysden, it was desirable to use the standard 101-key IBM keyboard configuration because users were familiar with it. Ex. 1114, 5:7-26. Further, during the prosecution of the 488 patent, the examiner found it inherent that a 101-key keyboard graphic keyboard can also be used (where the only difference was having more graphic objects ). Ex. 1103, Ex Page 72

73 C. Tsuji and Gettemy 6. Summary of Gettemy 143. U.S. Patent No. 9,489,018 to Gettemy et al. (Gettemy) was filed on November 4, 2009, and is a continuation of application No. 09/991,344, filed on November 20, I understand that Gettemy is a continuation of another patent application, and to the extent I rely on Gettemy, all of the subject matter relied on herein is in application No. 09/991,344, as filed on November 20, For the reasons discussed in this declaration, claim 25 is not entitled to a priority date of the earliest provisional application, November 1, 2001, and thus Gettemy is prior art to the 488 patent under 35 U.S.C. 102(e) (pre-aia). Gettemy is assigned to Qualcomm Inc. (Ex. 1112, cover), and is thus also prior art under 35 U.S.C. 103(a) Gettemy is directed towards providing a display with a touch sensor in a portable computer, such as a Personal Digital Assistant (PDA). Ex at Abstract, 3: As shown in Figure 5A below, Gettemy s PDA includes a front cover 425, a display device 212, a display device 211, optional backlight 550, and sensor 510. Ex at 7: Ex Page 73

74 146. Display device 211 can be an active or passive matrix LCD, an organic LED, or a variety of other display technologies. Id., 7: Figure 5B below shows what happens when the user opens the front cover (including display device 212) of PDA. 69 Ex Page 74

75 147. When the PDA is in an open state, the user can interact with display 211, which overlays sensor 510. Id., 8:66-9:10; see also 7:9-11 ( An optional backlight 550 is disposed beneath sensor 510 which is disposed beneath display device 211. ) (emphasis mine) Sensor 510 can be a capacitive sensor, such as the MultiTouch sensor which is commercially available from FingerWorks Incorporated of Newark, Del. Id., 8: Gettemy also discloses a stylus 415 that has a metal tip that can be detected by a capacitive sensor located below display device 211 and registered as an input to computer system 200. Id., 6: Ex Page 75

76 7. Statement of Rationale for Combination of JP725 and Gettemy 149. It would have been obvious to modify JP725 s information display device to have the display panel and touch panel configuration of Gettemy s PDA device. In such a configuration, JP725 s information display device would be modified so that the display panel 20 overlays touch panel 10T (or operating panel 10 in Figure 3), as shown in Gettemy. Thus, a user would contact the display panel, rather than the touch panel A POSITA would have been motivated to modify JP725 s information display device (for example the PDA embodiment) in a manner disclosed in Gettemy, which likewise discloses a PDA device with a touchscreen. Gettemy discloses that overlaying a display with a touchpad (as shown in JP725) can result in several problems, notably the parallax effect, which is a visual spatial distortion resulting in the location of a contact not corresponding to the intended target area of the LCD. Ex. 1112, 2: The effect is analogous to a stick being immersed in water, such that the stick takes on a bent or distorted appearance. Ex. 1112, 2: A POSITA would appreciate that JP725 s configuration as shown in Figure 3 could be improved accordingly A further drawback of the configuration where the touchpad overlays the display is that the amount of light from the LCD that is transmitted through the 71 Ex Page 76

77 touch screen and viewed by the user is diminished, and in some cases can be only about 80% of the available light. Ex. 1112, 3:1-6. In devices where a reflective display is used, the amount of light viewed by the user is further reduced to about 64%. Ex. 1112, 3:1-6. Thus, the configuration where the touchpad overlays the display reduces the overall contrast, clarity, and quality of the display as seen by the user. Id A POSITA would thus have been motivated to adopt a design whereby the display is overlaid on the touchpad to avoid issues arising from overlaying the touchpad over the display, such as the parallax effect, and to improve the quality of the images displayed on the touchscreen of JP725 s information display device. It would have been obvious to try to implement such a configuration using known methods and available technology, such as JP725 s expressly disclosed capacitive touchpad (discussed as an optional touch panel for the embodiment shown in Figure 16), and Gettemy s display (e.g., LCD display) to resolve display issues arising from the configuration shown in JP725. JP725 expressly discloses that touch panel 10T may include a capacitive touch panel. Ex. 1111, [0135]. 72 Ex Page 77

78 8. Claim 25 a. Limitation 25.pre: An apparatus comprising: 153. JP725 discloses an apparatus for the reasons described above for limitation [9.pre]. b. Limitation 25.a: at least one input device comprising a first position and a second position, the input device moveable to the first position upon application of a first pressure to the input device, and moveable to the second position upon application of a second pressure to the input device, the second pressure greater than the first pressure; 154. As discussed above for limitation [9.a], JP725 discloses an information display device. A POSITA would understand that JP725 s information display device, which includes a touch panel and a display panel, is an input device under the BRI for this term, and under the Board s prior construction for input device in IPR , a device by which a user can interact with an electronic device to provide instructions, responses, and other input to the electronic device. The touch panel and display panel enable the user to interact with the information display device in order to provide input (a location and force of an interaction with a button), and to receive a response (a haptic effect) JP725 discloses that the piezoelectric elements can consist of either ceramic piezoelectric elements or piezoelectric films. Ex. 1111, [0161]. In the 73 Ex Page 78

79 embodiment shown in Figure 23 below, JP725 s information display device uses piezoelectric films In this embodiment, piezoelectric films 310 are located beneath operating panel 10 (shown in Figure 3) or touch panel 10T (shown in Figure 15), and each film is supported by an elastic body 311 comprising a spring or other flexible material. Ex. 1111, [0161]. Although in this embodiment an LCD panel (not shown in Figure 23) is arranged under the operating panel 10 or touch panel 10T, JP725 in view of Gettemy discloses a configuration where the display overlays the touchpad. Id.. When a user applies force to the display panel, elastic bodies 311 contract, causing a voltage to be generated on both sides of the 74 Ex Page 79

80 piezoelectric film. Ex. 1111, [0161]. This voltage can be used to detect a pressing force and a pressing location. Id A POSITA would understand that the embodiment shown in Figure 23 would function identically to the embodiments shown in Figures 1-19, including the PDA and the ATM embodiments, except for the differences expressly stated in paragraphs [0160] and [0161]. JP725 supports this understanding because it expressly states that Figure 23 can use either operating panel 10 or touch panel 10T. Id. The voltage generated by the piezoelectric films would be used in a manner consistent with the voltages generated by the piezoelectric elements and received by the control circuit CT shown in Figures 7 and 16, or alternatively by the MPU in embodiments where circuit functions are performed by software instructions executed by the MPU and stored in a memory A POSITA would understand that, when a user contacts and applies force to the display panel, the elastic bodies 311 shown in Figure 23 will contract, resulting in movement of the display panel and the touch panel. Ex. 1111, [0161]. When a user applies a first pressure to the display panel and the touch panel, the elastic bodies will contract an amount that is proportional to the first pressure, and the touch panel will likewise move an amount proportional to the first pressure. Likewise, when a user applies a second pressure (greater than the first pressure), 75 Ex Page 80

81 the elastic bodies will contract further (i.e., in an amount proportional to the second and greater pressure), and the touch panel will likewise move an amount proportional to the second pressure. In embodiments where the elastic bodies 311 are springs, a POSITA would understand that the amount of contraction by the spring is directly proportional to the amount of force applied to the spring under general physics principles As discussed above in the context of claim 9, JP725 discloses using a capacitive operating/touch panel for the embodiment shown in Figure 16. Ex. 1111, [0134]-[0135]. The modification of moving the display panel to overlay the operating/touch panel (consistent with the teachings of Gettemy) would not have a meaningful difference in the voltage signals output based on the amount of contraction of elastic bodies 311 that are connected to piezoelectric films 310. The user would continue to interact with the operating surface (display panel) by applying pressure to a button (e.g., the Withdrawal button shown in Figure 4), which would result in the display panel and touch/operating panel moving to different positions (based on contraction of the springs), where that movement includes a first position and a second position. 76 Ex Page 81

82 c. Limitation 25.b: at least one actuator in communication with the input device, the actuator configured to output tactile sensations to the apparatus; and 160. See limitation [9.b]. A POSITA would understand that, in the context of the embodiment shown in Figure 23, the piezoelectric films would output tactile sensations to the apparatus, including the touch/operating panels, display, and case 40. d. Limitation 25.c: at least one processor in communication with the input device, the processor configured to: 161. See limitations [9.c] and [25.a]. As discussed above for preamble [9.pre] and limitations [9.a] (sensor) and [9.b] (actuator), the ATM and PDA embodiments use the same primary variations of control circuit CT In the first variation, JP725 discloses that the processor (comprising the components outside the dashed line in annotated Figure 7 below) is in communication with the input device (including sensor and actuator) through drive mode selecting unit 72, which is part of the processor for the reasons explained in limitation [9.c]. Ex. 1111, Figs. 7, 16, [0108]. In other words, in software embodiments, unit 72 (and the logic contained therein) is part of JP725 s MPU. The second variation discloses this limitation for the same reason (i.e., the 77 Ex Page 82

83 components outside the dashed line are similar, if not identical, as shown in Figure 16). e. Limitation 25.d: receive a first input signal from the input device, the first input signal associated with the first position, 163. As discussed for limitation [25.a] (inn the embodiment shown in JP725 s Figure 23), the contraction of elastic bodies 311, based upon a force applied by a contact to the surface of the information display device, causes voltage signals e1-e4 to be output and received by JP725 s processor. Those voltage signals are used to generate signals SP (position) and SF (force). Signal SF is input into operating force determining unit 52,, which outputs signal FB to unit 72 (which is part of the processor, as explained above for limitation [9.c]). 78 Ex Page 83

84 164. The voltage signals output by piezoelectric films are proportional to the amount of contraction of elastic bodies 311, and a POSITA would understand that the signal SF generated by operation unit 51/51F and the signal FB generated by the operating force determining unit 52 are associated with the voltage signals that are output from the piezoelectric films (and the amount of contraction of elastic bodies 311) For example, when a user applies a first pressure to the surface of the information display device (that is greater than a first pressure threshold Fh1), the elastic bodies 311 (and connected components) will contract to a first position, and piezoelectric films will output voltage signals proportional to the contraction of elastic bodies 311. Those voltage signals are then used to generate operating force signal SF and then FB. f. Limitation 25.e: receive a second input signal from the input device, the second input signal associated with the second position, 166. See limitation [25.d]. As a further example, when a user applies a second pressure to the surface of the information display device (that is greater than a second pressure threshold Fh2), the elastic bodies 311 (and connected bodies) will contract to a second position, and piezoelectric films will output voltage signals proportional to the contraction of elastic bodies 311. Those voltage signals are then used to generate operating force signal SF and then FB. 79 Ex Page 84

85 g. Limitation 25.f: transmit a first actuator signal to the actuator, the first actuator signal configured to cause the actuator to output a first tactile sensation associated with the first position, and 167. See limitations [9.g], [9.h], and [25.f]. As discussed above for limitations [25.d] and [25.e], the voltage signals output from piezoelectric films 310 (and the resulting signal SF and signal FB) are associated with a first position. As discussed above for limitations [9.g] and [9.h], signal FB (based on signal SF) is used to determine a pressure range F0, F1, F2, F3, and F4, and based on the region selected by the user and the amount of pressure detected, the drive mode selecting unit will select an actuator signal (V) to output a tactile sensation to piezoelectric films to the drive unit A POSITA would understand that, when the user applies force to the surface of JP725 s information display device, this will result in voltage signals being output that are associated with a first position (based on the amount of contraction of elastic bodies 311), and likewise will result in the generation of signal FB (based on signal SF) that is used to determine the actuator signal to output to the actuator. A POSITA would further appreciate that the actuator signal is associated with the first position because it is derived from the voltage signals output by piezoelectric films, which are associated with the first position as discussed above. 80 Ex Page 85

86 169. For example, in the scenario where the user selects region R 1, and when the first pressure applied by the user results in a pressure range F1, and the second pressure applied by the user results in a pressure range F2, the first actuator signal will be S 11, and the second actuator signal will be S 12. Ex. 1111, Fig. 13. The actuator signals S 11 and S 12 cause the first tactile sensation and the second tactile sensation, and a POSITA would understand that those sensations could either be the same (or different) from each other. Examples of signals causing varying continuous vibration (14(a) 14(c)) and burst vibration (14(d) 14(f)) are shown in Figure 14 below. 81 Ex Page 86

87 h. Limitation 25.g: transmit a second actuator signal to the actuator, the second actuator signal configured to cause the actuator to output a second tactile sensation associated with the second position; 170. See limitation [25.f]. As an example, if the user applies a pressure corresponding to F2 (where the region selected is R 1 ), then actuator signal S 12 will be output based on Figure 13. A POSITA would understand that when the user applies the second pressure to the surface of JP725 s information display device, this will result in voltage signals being output that are associated with the second 82 Ex Page 87

88 position, and likewise will result in the generation of operating force signal SF that is used to determine the actuator signal to output to the actuator. i. Limitation 25.h: wherein the input device comprises a pressure-sensitive touchpad, and the apparatus further comprises: 171. See limitations [9.a] and [25.a]. j. Limitation 25.i: a display panel in communication with the pressure-sensitive touchpad, the display panel configured to receive the tactile sensations from the pressure-sensitive touchpad; and 172. When JP725 s information display device is modified in a manner described above to include a display panel overlaying the touch panel (as discussed in Gettemy), the display panel is in communication with the touch panel (pressuresensitive touchpad) because it is physically connected to the touch panel. Thus, the display panel is in communication with the pressure-sensitive touchpad in JP725 s modified information display device Further, when drive unit 75 (shown in Figures 7 and 16) outputs a drive signal to the piezoelectric films (shown in Figure 23), this will cause the piezoelectric films to output a vibration that causes the surface of the information display device to vibrate. Ex. 1111, Abstract, Figs. 3, 15, 23. Specifically, the piezoelectric films will cause vibrations that will be communicated to the touch panel because the films are located under the touch panel, and a POSITA would 83 Ex Page 88

89 understand that those vibrations will be communicated to the display panel through the touch panel. Thus, in such a configuration of JP725 s information display device, that combination renders obvious this limitation. k. Limitation 25.j: at least one software-generated button configured to be displayed on the display panel See limitation [9.d]. In the ATM embodiments, a POSITA would understand that the buttons shown on the ATM menu, for example the Withdrawal button and the Deposit button, are generated by software stored in memory and accessed by JP725 s processor. And in the PDA embodiment, a POSITA would understand that the buttons shown on the PDA screen, for example the R 1, R 2, R 3, and R 4 buttons, are generated by software stored in memory and accessed by JP725 s processor Further, as shown in Figures 7 and 16, the processing unit outputs a display signal to LCD panel 20 through display driver 71. Thus, a POSITA would understand that the buttons displayed in the ATM and PDA embodiments are software-generated, and are further configured to be displayed on Gettemy s flexible display panel comprising electronic paper technology that uses liquid crystals. As discussed above, both JP725 and Gettemy disclose display screens that use liquid crystal technology. 84 Ex Page 89

90 9. Claim 26: The apparatus of claim 25, wherein the touchpad comprises a pressure calculator to measure the distinct amount of pressure JP725 discloses an operation unit 51 (shown in Figure 7) and operation unit 51F (shown in Figure 16) that receives voltage signals e1-e4 from piezoelectric elements E1-E4 and outputs an operating force signal SF. In those embodiments, a POSITA would understand that the operation unit 51/51T determines the distinct amount of pressure applied by the user to the surface of JP725 s information display device under the BRI. This pressure measurement is then used to determine the pressure range, as discussed above. Further, a POSITA would understand that, as shown in annotated Figure 7 (see limitation [9.a]), these components are part of the touchpad. D. Tsuji, Gettemy, and Westerman 1. Summary of Westerman 177. U.S. Patent No. 6,323,846 to Westerman et al. ( Westerman ) was filed on January 25, 1999, and is prior art under 35 U.S.C. 102(e). Westerman discloses a system for tracking finger and palm contacts as a user touches a flexible multi-touch surface. Ex. 1113, Abstract. The multi-touch surface has various layers, including an electrode array, that enables low-noise proximity images to be generated. Id. 85 Ex Page 90

91 2. Statement of Rationale for Combination of JP725, Gettemy and Westerman 178. JP725 expressly discloses that the variation shown in Figures 15 and 16 uses a capacitive touch panel, and alternatively could use various types of touch sensors well known in the art. Ex. 1111, [0135] Westerman is an example of one such prior art capacitive touch panel. The named inventor, Wayne Westerman, is also named as an inventor on many patent publications and issued patents that are (or were) assigned to FingerWorks, the same company expressly identified in Gettemy and identified as a manufacture of commercially available capacitive touch sensors, such as the MultiTouch sensor. This is shown in the assignment history for Westerman, which identifies Fingerworks as an assignee A POSITA would thus would appreciate that Gettemy s touch sensor could use the capacitive touch sensor disclosed in Westerman. Indeed, a POSITA would have been motivated to review FingerWorks publications and product offerings, particularly in a device using a capacitive touch sensor, based on the express teaching in Gettemy to use the FingerWorks MultiTouch sensor. This express teaching would have led one of ordinary skill in the art to Westerman. In 2 Available at &type=patNum. 86 Ex Page 91

92 addition to the express motivation provided by Gettemy, one of ordinary skill in the art would have been motivated to use the multitouch sensor in Westerman to enable different modes (stylus v. finger) without requiring explicit mode switching, multitouch support, capacitive sensing through thick layers (like an LCD) and the many other advantages as described in Westerman itself. Ex at 1:49-59, 3:2-6, 3:48-51, 3:67-4:3, 4:20-28, 5:25-30, 6:6-9, 6:50-52, 6:55-7: In such a combination, JP725 s MPU would continue to receive an output from the piezoelectric elements, and would additionally receive an output from Westerman s touch sensor (used for Gettemy s capacitive touch sensor). As discussed above, JP725 discloses that the operating position specification unit 51T (in the second variation) receives signals from the resistive touch panel. Ex. 1111, Fig. 16, [0139]. Thus, a POSITA would understand how to incorporate the output from a resistive touch sensor (or any of the other touch sensors expressly disclosed) in JP725 s information display device, as needed in order for JP725 s hardware and/or processor to receive signals from the touch sensor. JP725 already discloses providing signal lines for position information received from the touch sensor shown in Figure 16, and it would have been obvious to a POSITA to add an additional signal line, or use the existing signal line to transmit both 87 Ex Page 92

93 information data obtained from the touch sensor, for example for purposes of determining the pressure value output by the touch sensor For the reasons below, it is my opinion that JP725 in view of Gettemy and Westerman disclose claims 27 and Claim 27 a. Limitation [27.a]: The apparatus of claim 25, further comprising: an interface object configured to be used to contact a location on the display panel corresponding to the at least one software-generated button; and 183. Gettemy discloses a stylus that has a metal tip that can be detected by a capacitive sensor located below display device 211 and so registered as an input to computer system 200. [Ex. Gettemy], 6: Westerman similarly discloses that using a stylus to handwrite on a touch surface was well-known in the art. See, e.g., Ex. 1113, 4:20-21 ( [h]andwriting on smooth touch surfaces using a stylus is well-known in the art ), 41:31 ( conductive stylus ), claim 97 ( The method of claim 95, wherein a stylus held between the pinched fingers touches the surface instead of the pinched fingers themselves ) A POSITA would understand that the stylus device disclosed in Gettemy and Westerman are both an interface object under the BRI for that term. This is consistent with dependent claim 28, which requires that the claimed interface object comprises a stylus. A POSITA would further understand that a 88 Ex Page 93

94 stylus, such as those disclosed in Gettemy and Westerman, could be used to contact the surface of JP725 s information display device, including a location corresponding to the at least one software-generated button, especially when the information display device has been modified to include Gettemy s display and Westerman s capacitive touch technology to increase the accuracy and brightness of the display as taught by Gettemy An intended use of such a device would be to interact with a button, for example the Withdrawal button on the ATM menu (shown in Figure 4) or the Up or Down buttons on the interface of Tsuji s PDA embodiment. This would enable the user to access an ATM menu to specify an amount of money to withdraw from the ATM device, or to manipulate a graphical object, as shown for example in Figures 17 and 18 of the PDA embodiment. b. Limitation [27.b]: a pressure calculator to calculate an amount of pressure based upon an amount of area of the interface object in contact with the display panel JP725 in view of Gettemy discloses a pressure calculator for the reasons explained for claim 26, and JP725 in view of Gettemy and Westerman discloses a stylus (interface object) that is used to contact Gettemy s display (with the capacitive touch sensor underneath). See claim 26 and limitation [27.a]. 89 Ex Page 94

95 187. Westerman discloses that calculating an amount of pressure (application of force from a contact) would be based upon an amount of area of the interface object (stylus) in contact with the display panel. Westerman discloses that the conductive stylus would take the place of a fingertip, and the index fingertip path sent to the host communication interface will in actuality be caused by the stylus. Ex at 41: Westerman discloses measuring pressure between a touch device such as a finger and the surface 2, which it also refers to as proximity, and which is interpreted as pressure in the z-axis. Ex at 14: Westerman discloses numerous embodiments for measuring proximity and pressure. Ex at Figs. 2-12, 14:45-18:12. Westerman discloses the sensor can sense both proximity and pressure of a touch device by measuring the distance moved, which depends on the pressure applied. Ex at Fig. 12, 18:7-12. This includes calculating the proximity (pressure) based on area, because it integrates the sensors under the area of contact, which is a calculation of pressure based on area. Ex at Fig. 18, 23:21-24:34, 26:7-27:18. The higher level modules use the proximity (pressure) to distinguish finger (stylus), palm and thumb contacts. Ex at 26:4-6. Westerman also discloses using the area of contact of fingers as part of the calculation of pressure. Ex at 19:7 ( index finger 202 ), 23:32-90 Ex Page 95

96 54; 31: Westerman teaches that a stylus is treated as the index finger. Ex at 41: Thus, when Tsuji s information display device is modified to include the configuration shown in Gettemy, using Westerman s multitouch capacitive sensor as expressly taught by Gettemy and disclosed in Westerman, a POSITA would understand that the capacitive touch sensor generates a measurement reflecting an amount of area of the interface object in contact with the display panel. i. Claim 28: The apparatus of claim 27, wherein the interface object comprises a stylus JP725 in view of Gettemy and Westerman disclose claim 28 for the reasons provided for claim 27. V. CONCLUSION 190. For the foregoing reasons, it is my opinion that claims 3-8, 11-16, and are invalid and should be canceled. 91 Ex Page 96

97 I declare that all statements made herein of my own knowledgee are true and that all statementss made on informatio on and belief are believed to be true, and further that these statements s were made with the knowledge that willful false statements and the like so made are punishable by fine or imprisonment, or both, under Section 1101 of Title 18 of the United States Code. Executed on May 4, Dr. Majid Sarrafzadeh 92 Ex Page 97