Final Report: Design and Implementation of a Binary Neural Network

Transcription

1 Smith & Maitra 1 Final Report: Deign and Implementation of a Binary Neural Network Daniel Smith & Shamik Maitra E158 Intro to CMOS VLSI Deign Prof. Harri

2 Smith & Maitra 2 Background: In a typical neural network, a et of input i operated on by a erie of nonlinear element called neuron. Thi network of neuron produce a et of output baed on thee input. Neural network have many practical application, which motly tem from the fact that the trength of a neural network i it ability to identify alient feature from large data et. Typical application are data compreion, facial recognition, trend analyi/prediction, etc. The neuron are uually arranged in architecture that upport the function that they are expected to perform. On a baic level each neuron connection to another neuron or input carrie with it ome trength, or weight. Whenever an input arrive at a neuron, it i multiplied by thi connection weight. Typical neuron (in the computational ene) multiply each input by a connection weight and add up all of thee weighted input. Thi um i uually ent through an activation function to determine whether or not a neuron will fire (or to what degree). While uch network exit primarily a oftware and computer model, ome have been hardwired into circuit a well a IC. Several neural net chip exit on the market today. Some of thee chip operate a analog device by running below threhold on the tranitor thereby gaining continuou propertie intead of dicrete propertie afforded by CMOS tranitor logic. Other chip, however, do ue purely the digital capacity of CMOS to execute a network; and yet other ue hybrid between the two to create a more robut deign.

3 Smith & Maitra 3 Functional Overview: We accomplihed the deign and implementation of our neural network through a good degree of analogizing procee that take place in a neuron to procee that we felt we could replicate efficiently on a chip. We felt efficiency, in term of pace, wa important becaue we wanted to fit a many neuron a poible onto thi chip. Since we wanted our neural network to be able to be ued for different application, we decided to fully connect each layer to the next layer. Thi mean that each of the five input to the network i connected to each of the 5 neuron in the firt layer. To accomplih the weighting function performed by typical neuron, we decided to perform logical operation on each of the input to the neuron with known value (weight, in other word). We accomplihed thi by toring our binary connection weight in flop. We decided to implement thee connection weight at the individual neuron becaue it eliminated the need to have one central memory holding all weight for the whole network. Each input to each neuron ha two weight aociated with it. The firt weight i NAND-ed with the input, and the output of thi operation i XOR-ed with the econd weight.

4 Smith & Maitra 4 The reaon for including both of thee operation i that they provide u with more flexibility in how the neuron deal with the input. The above chematic how the hardware we are uing to weight each input to each input. The following truth table illutrate the advantage of uing two weight intead of one. Weight1 Weight0 Input Output Vdd Gnd Wire A you can ee, each independent et of two weight pae a different input to the neuron activation function. Our neuron activation function i the NOR5. Each neuron weight each of it five input eparately and then NOR5- them all together. Now, given the above cheme for the four different poible weight that an input can be ubject to, we can develop ome inight a to how all the neuron hould be weighted in order to produce a deired output from a given input.

5 Smith & Maitra 5 For example, ince the NOR i only high when all input to it are low, if we want to tell it to ignore an input, we tie the weighter correponding to that input to ground (not phyically but by giving it a weight of 01). Another example would be if we wanted to turn a neuron into a NOR5, we would jut give all the weighter in the neuron the weight 11. Thi would tell the weighter to pa all five input to the neuron on to the NOR5 at the end. More detail about thi functionality are given in the imulation ection. So far we have ignored how the weight are given to the neural network. Each flop (toring one weight) in addition to having it output connected to a logic gate alo ha it output connected to the next flop. In thi way, all of the flop all over the chip form a ditributed hift regiter. The firt flop on the chip will accept the weight to be hifted in, and a the chip (hift regiter) i clocked, the data will move through until all 150 weight have been hifted into the chip. Since the clock i pread over almot the entire chip, we are uing 2-phae clocking with no overlap to enure that all weight are properly hifted in without any data corruption. The downide of thi i that to change a ingle weight you mut hift in all the weight again, however, ince there are only 150 weight, thi houldn t be too laboriou uing any ort of controller with a clock rate of a couple of hertz or more. In hort our deign ue everal abtraction from a continuou model of neural network. Through appropriate choice for our weight we can perform variou function with our neuron. Our network conit of 3 layer (tack of 5 neuron) that are fully connected in order. That i, the input are fully connected to layer 1; layer 1 i fully connected to layer 2; etc. Each neuron output i the NOR of the weighted input. The weight are input to the chip by mean of a ditributed hift regiter.

6 Smith & Maitra 6 Chip Floorplan: 7λ 7λ 150λ 92λ Weighter 1 Weighter 2 Weighter 3 Weighter 4 Weighter 5 NOR5 Layer λ x 1952λ Layer 3 307λ Neuron λ x 385λ Neuron 3 Neuron 4 Neuron 5 8λ Ring Ocillator 162.5λ x 77λ 90λ Thi floor plan exclude the pad frame for the chip. The outer dimenion of thi floorplan include all of the facet in the core of our chip. Thi floorplan how the bigget facet in the core. The firt layer i diected to how the neuron, and the firt neuron i diected to how the component. The weighter i laid out in a data path tyle. Within the data path are two flop, one NAND, and one XOR, which are not hown. Alo, within the flop are two latche clocked eparately. Thee are alo not hown to keep the floorplan readable.

7 Smith & Maitra 7 Chip Pinout: Power Vdd Input ph_1 Input ph_2 Input Data in Input In 1 Input In 2 Output Out 1 Input In 3 Output Out 2 Input In 4 Output Out 3 Input In 5 Output Out 4 Output Out 5 Input ring enable Output Ring output Ground Gnd Output Data Out Decription: To input weight put the weight (lat weight firt) on Data in while clocking ph_1 and ph_2. The five input to the network are input In 1 In 5. The output of the weight hift regiter come on data out. The five output of the network are Out 1 Out 5. At the bottom, a mall ring ocillator i provided for tet purpoe. Set ring enable to high to turn on the ocillator. We expect the frequency of the ocillator be around 230 MHz.

8 Leaf Cell Detail: XOR Smith & Maitra 8

9 Smith & Maitra 9 LATCH NAND2

10 FLOP Smith & Maitra 10

11 Flop Layout Smith & Maitra 11

12 Smith & Maitra 12 PSEUDO_NOR5 Decription: Thi facet i a peudo nmo gate. There one weak pullup nmo tranitor, far left, which i alway on. The ret of the tranitor are four time it ize. If any of them turn on, the output (left ide) will go low, making a NOR gate.

13 Smith & Maitra 13 Weighter Decription: Thi facet take two ph_1 clock and two ph_2 clock to match the layout. The layout had two clock o that each weighter can nap together with the weighter above, making the clock run a four parallel line over each layer. The input i exported on the far left in the middle, data in on the left at the top, and data out on the left at the bottom. The output i exported on the right.

14 Neuron Smith & Maitra 14

15 Neuron Layout Smith & Maitra 15

16 Layer Smith & Maitra 16

17 Smith & Maitra 17 Layer Layout Note: Thi i rotated to be horizontal. The input come in the top and the output come out the bottom. It i five neuron tacked.

18 Network Smith & Maitra 18

19 Network Layout Smith & Maitra 19

20 Ring Ocillator Smith & Maitra 20

21 Core Smith & Maitra 21

22 Core layout Smith & Maitra 22

23 Top Level Smith & Maitra 23

24 Smith & Maitra 24 Summary of Deign: Cell Actual Actual Tranitor Area / Deign Time Comment Cell Size Cell Area Count Tranitor Sch (hr.) Lay (hr.) Tet (hr.) 1td_latch 80 x 77 6, modification of tandard cell to ue metal 2 a horizontal 2flop 169 x 77 13, uing two modified td_latche back to back to form a flop 3td_nand2 33 x 77 2, modification of tandard cell to ue metal 2 a horizontal 4XOR 85.5 x 77 6, ome layout work required to keep within floorplan etimate 5Weighter x 77 33, quite a lot of layout work to enure nap-together-ne 6Pudo-NOR x , weird ize and hape to fit the height of the whole neuron 7Neuron x , alo had to enure the ability of neuron to nap together 8Layer x ,189, pretty quick, involved zipping together neuron 9Network 1841 x ,683, three layer placed ide by ide, with a few connection 10Ring Ocillator x 77 12, Core 1841 x ,841, Top 2754 x Total: Total Deign Time: 39 hr. Thi i the ummary of how much time we pent actually working with electric for the chip. We actually pent quite a bit of time on developing the concept of how we could bet perform the function of a theoretical neural network given our limited hardware, experience, and time. We were unable to actually figure out the tranitor count of the pad frame and o thoe entrie are left blank on our table. Part of what contribute to the lightly high (for a datapath) area per tranitor i the fully connected nature of our network. Thi involved quite a few big wire, which do wonder at increaing the area of our layout without changing our tranitor count.

25 Smith & Maitra 25 Simulation Detail: Input Pattern Output Pattern Reult I: Thi i the graphical output of the hifttet cript (ee appendix) on the core layout. None of the aertion in hifttet failed. Thi tet fill all 150 flop in the network with random value. It then clock the chip 150 time to get out all the value again and check to make ure that they are all correct. You can ee that the output pattern (left half of the imulation, matche in the input pattern (right half of the imulation. Thi indicate that our hift regiter at leat i working properly. Input Output Reult II: Thi i the output of the identtet1 cript on the core layout. Thi tet hift in a pattern of weight which make the identity network, each output value equal it correponding input value. Prior to the waveform above, the 150 bit were hifted in.

26 Smith & Maitra 26 Here we can ee that after the hifting, the output are indeed matching the input to the network. None of the aertion in idnettet1 failed on thi tet. There are five different identity tet, each one of which rotate the output by a different amount. Identtet2 rotate the output by 3 (1 rotation per layer).

27 Smith & Maitra 27 Deign Verification: Cell Sub-Cell Complexity Simulate DRC ERC NCC Etimated Actual Cell Size Cell Size 1td_latch - N/A x x x x 80 x x 77 2flop 1 1 x x x x 80 x x 77 3td_nand2 - N/A x x x x 80 x x 77 4XOR - 3 x x x x 80 x x 77 5Weighter 1,2,3,4 4 x x x x 80 x x 77 6PudoNMOS NOR5-2 x x x x 80 x x Neuron 5,6 5 x x x x 400 x x 385 8Layer 7 4 x x x x 2000 x x Network 8 3 x x x x 2000 x x Ring Ocillator - 2 XXX x x x XXX x 77 11Core 9,10 2 x x XXX Gem XXX 1841 x Top 10,11 3 XXX XXX XXX XXX 2754 x 2754 The different component of our ytem all imulate fine individually and all pa DRC, ERC, and NCC. However, the ring ocillator, which we placed on the chip a a tet tructure to be ued once the chip i fabricated doen t pa ERC when placed in the overall core of the chip becaue of it placement. Since it i crammed beneath our network, it didn t make ene to connect it to the power and ground in the core-level layout. We connected it, intead, to power and ground in the top-level chematic where power and ground are needed to enable the appropriate input and output pad. Since power and ground were not connected to Network power and ground in the core, ERC aw the P well and N well not connected to any ource of power or ground to properly bia them againt leakage into the ubtrate. Hence the ring ocillator made our overall core layout fail ERC. It alo failed Electric NCC however, Gemini aid that it wa equivalent to the chematic we provided for thi layer.

28 Smith & Maitra 28 Additionally, the ring ocillator wa unable to imulate properly on IRSIM. We believe that the reaon for thi i that it might be a difficult circuit for IRSIM to analyze in term of timing, ince it i really a loop that doen t end and i dependent on the tate of the ocillator ytem at any given time. Another point of interet i the top-level cell (facet). It failed DRC becaue the pad frame i imported from Caltech Interchange Format to Electric and i hence decribed in term of pure layer. Thi doe not fit with Electric chema for performing DRC. ERC wa never able to go through to completion without crahing Electric on the Top-level layout, and NCC wa impoible to run a well ince we were not provided with a chematic for the pad frame. When we propoed the project, we hadn t propoed to put a ring ocillator on the core and o there i no etimated ize for thi. The ize of the top-level layout wa what we were deigning for, and our deign fit comfortably inide it with enough room to inert a tet tructure uch a our ring ocillator. On the bai of imulation and the checker in electric and IRSIM, however, we are fairly confident that our deign i ound and will be able to perform it function in a atifactory manner.

29 Smith & Maitra 29 Tet Plan: The plan for the teting procedure of our chip after fabrication i a follow: 1. Ue the ring ocillator tructure on the chip with the enable tied to Vdd and meaure the output with an ocillocope. We expect to ee a (probably deformed) quare wave output at approx 230 MHz. We are uing a 9-tage ocillator that ha a pretty high frequency, but it i till within the upper limit of the meauring capacity of the ocillocope available to u in the electronic lab at Harvey Mudd College. 2. Upon confirmation that the chip i not defective (a confirmed by tep 1), the hift regiter functionality hould then be teted by phyically performing the function of the program hifttet which i included in Appendix A. Shifttet hift into the large regiter a random equence of 150 weight, and tet them to make ure that they are hifted out properly and uncorrupted. 3. Once the functionality of the hift regiter i aured, the firt tep will be to try to replicate the reult of the program identtet1 (ee Appendix A). Thi program hift in the appropriate weight to allow the network to replicate an input pattern. It hould be noted that the chip i entirely combinational once the weight have been hifted in. The clock ole purpoe i for the purpoe of moving data through the hift regiter. Thi i a good tarting point becaue it i a relatively imple pattern to diagnoe trouble in individual neuron circuitry. If the given input vector doen t match the given output vector, there i a problem. The neuron, but not the layer at fault can be pinpointed uing one-hot encoded vector. Further

30 Smith & Maitra 30 application of the proper weight and vector hould allow the teter to diagnoe the neuron/layer at fault. If neceary, weight in other neuron may be adjuted to accommodate the failure of a few neuron. We can treat the output of the neuron a an ignore by weighted it repective input at all connected neuron with Once appropriate neuron are determined to be functional, executing 1 any of the following program on the chip will more thoroughly tet functionality of the combinational logic on the chip: identtet2, identtet3, identtet4, identtet5 (ee Appendix A). Thee all caue each layer to barrel hift the input pattern by a fixed amount. Identtet2 barrel hift the input by 1 place at each layer for a total hift of three lot. Likewie, identtet3 barrel hift the input pattern by 2 place at each layer for a total of 6 place. Thi pattern of incrementing the hift amount per layer continue through identtet5. 5. Finally, upon ucceful completion of thee tet, we may look at more logically complex function. It hould not be hard to write a program to teach the neural network to accomplih a tak uing a Boltzmann learning algorithm. Thi involve providing an input and deired output, and randomly witching weight to decreae the overall Energy function of the ytem. More invetigation i definitely required to do thi kind of training. However, provided in Appendix B i a Matlab decription of a Neural Network that wa written to decribe thi particular network. So, if one wanted to imulate the reult provided by uch a training program without actually having to load

31 Smith & Maitra 31 weight onto the chip, they hould run the et of weight given by their training program into the provided Matlab code and try different tet vector to ee whether or not the training program ha given an accurate et of weight to perform the particular application. 1 It hould be noted that by program execution, we mean to actually hook up a proceor to hift in the appropriate weight given in the program and then phyically providing the neceary input to the chip and meauring the output.

32 Smith & Maitra 32 Appendix A:Tet File inv l Network_data_in c h Network_data_in c buff l Network_data_in c l Network_data_in c one h Network_data_in c l Network_data_in c zero h Network_data_in c h Network_data_in c checkbuff aert Network_data_out 1 c aert Network_data_out 1 c checkinv aert Network_data_out 0 c aert Network_data_out 1 c checkone aert Network_data_out 0 c aert Network_data_out 0 c checkzero aert Network_data_out 1 c aert Network_data_out 0 c initclock clock ph_ clock ph_ hifttet

33 Smith & Maitra one

34 Smith & Maitra checkinv

35 Smith & Maitra checkbuff

36 Smith & Maitra checkinv layerident layerident

37 Smith & Maitra layerident layerident

38 Smith & Maitra layerident layerident1

39 Smith & Maitra 39 layerident2 layerident3 layerident4 layerident5 netident1 l Network_in_1 Network_in_2 Network_in_3 Network_in_4 Network_in_5 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 h Network_in_1 aert Network_out_1 1 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 h Network_in_2 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 h Network_in_3 aert Network_out_1 1 aert Network_out_2 1

40 Smith & Maitra 40 aert Network_out_3 1 aert Network_out_4 0 aert Network_out_5 0 h Network_in_4 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 0 h Network_in_5 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 1 l Network_in_1 aert Network_out_1 0 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 1 l Network_in_2 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 1 l Network_in_3 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 1 aert Network_out_5 1 l Network_in_4 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0

41 Smith & Maitra 41 aert Network_out_4 0 aert Network_out_5 1 l Network_in_5 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 l Network_in_1 Network_in_3 Network_in_5 h Network_in_2 Network_in_4 aert Network_out_1 0 aert Network_out_2 1 aert Network_out_3 0 aert Network_out_4 1 aert Network_out_5 0 h Network_in_1 Network_in_3 Network_in_5 l Network_in_2 Network_in_4 aert Network_out_1 1 aert Network_out_2 0 aert Network_out_3 1 aert Network_out_4 0 aert Network_out_5 1 netident2 l Network_in_1 Network_in_2 Network_in_3 Network_in_4 Network_in_5 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 h Network_in_1 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 1 aert Network_out_5 0 h Network_in_2

42 Smith & Maitra 42 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 1 aert Network_out_5 1 h Network_in_3 aert Network_out_1 1 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 1 aert Network_out_5 1 h Network_in_4 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 0 aert Network_out_4 1 aert Network_out_5 1 h Network_in_5 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 1 l Network_in_1 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 0 aert Network_out_5 1 l Network_in_2 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 0 aert Network_out_5 0 l Network_in_3 aert Network_out_1 0

43 Smith & Maitra 43 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 0 aert Network_out_5 0 l Network_in_4 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 1 aert Network_out_4 0 aert Network_out_5 0 l Network_in_5 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 l Network_in_1 Network_in_3 Network_in_5 h Network_in_2 Network_in_4 aert Network_out_1 0 aert Network_out_2 1 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 1 h Network_in_1 Network_in_3 Network_in_5 l Network_in_2 Network_in_4 aert Network_out_1 1 aert Network_out_2 0 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 0 netident3 l Network_in_1 Network_in_2 Network_in_3 Network_in_4 Network_in_5 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 h Network_in_1

44 Smith & Maitra 44 aert Network_out_1 0 aert Network_out_2 1 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 h Network_in_2 aert Network_out_1 0 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 0 aert Network_out_5 0 h Network_in_3 aert Network_out_1 0 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 0 h Network_in_4 aert Network_out_1 0 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 1 h Network_in_5 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 1 l Network_in_1 aert Network_out_1 1 aert Network_out_2 0 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 1 l Network_in_2

45 Smith & Maitra 45 aert Network_out_1 1 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 1 aert Network_out_5 1 l Network_in_3 aert Network_out_1 1 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 1 l Network_in_4 aert Network_out_1 1 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 l Network_in_5 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 l Network_in_1 Network_in_3 Network_in_5 h Network_in_2 Network_in_4 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 1 aert Network_out_4 0 aert Network_out_5 1 h Network_in_1 Network_in_3 Network_in_5 l Network_in_2 Network_in_4 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 0 aert Network_out_4 1 aert Network_out_5 0 netident4

46 Smith & Maitra 46 l Network_in_1 Network_in_2 Network_in_3 Network_in_4 Network_in_5 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 h Network_in_1 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 1 h Network_in_2 aert Network_out_1 1 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 1 h Network_in_3 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 1 h Network_in_4 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 0 aert Network_out_5 1 h Network_in_5 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 1

47 Smith & Maitra 47 l Network_in_1 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 0 l Network_in_2 aert Network_out_1 0 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 0 l Network_in_3 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 0 l Network_in_4 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 1 aert Network_out_5 0 l Network_in_5 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 l Network_in_1 Network_in_3 Network_in_5 h Network_in_2 Network_in_4 aert Network_out_1 1 aert Network_out_2 0 aert Network_out_3 1 aert Network_out_4 0 aert Network_out_5 0 h Network_in_1 Network_in_3 Network_in_5 l Network_in_2 Network_in_4

48 Smith & Maitra 48 aert Network_out_1 0 aert Network_out_2 1 aert Network_out_3 0 aert Network_out_4 1 aert Network_out_5 1 netident5 l Network_in_1 Network_in_2 Network_in_3 Network_in_4 Network_in_5 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 h Network_in_1 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 1 aert Network_out_4 0 aert Network_out_5 0 h Network_in_2 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 0 h Network_in_3 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 1 h Network_in_4 aert Network_out_1 1 aert Network_out_2 0 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 1

49 Smith & Maitra 49 h Network_in_5 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 1 aert Network_out_5 1 l Network_in_1 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 0 aert Network_out_4 1 aert Network_out_5 1 l Network_in_2 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 1 l Network_in_3 aert Network_out_1 1 aert Network_out_2 1 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 l Network_in_4 aert Network_out_1 0 aert Network_out_2 1 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 l Network_in_5 aert Network_out_1 0 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 0 aert Network_out_5 0 l Network_in_1 Network_in_3 Network_in_5

50 Smith & Maitra 50 h Network_in_2 Network_in_4 aert Network_out_1 1 aert Network_out_2 0 aert Network_out_3 0 aert Network_out_4 1 aert Network_out_5 0 h Network_in_1 Network_in_3 Network_in_5 l Network_in_2 Network_in_4 aert Network_out_1 0 aert Network_out_2 1 aert Network_out_3 1 aert Network_out_4 0 aert Network_out_5 1 hifttet print ******************************************************** print hift tet complete print identtet1 print ******************************************************** print identtet1 complete print identtet2 print ******************************************************** print identtet2 complete print identtet3 print ******************************************************** print identtet3 complete print identtet4 print ******************************************************** print identtet4 complete print identtet5 print ******************************************************** print identtet5 complete print ********************************************************

51 Smith & Maitra 51 Appendix B: Matlab Simulation We wrote a imulation of our network in matlab. The hope wa that we would be able to train thi imulation in matlab and then write the learned weight onto the chip, thu getting around the fact that our chip ha no hardware learning. Unfortunately, we were not able to eaily adapt any of matlab neuron net training function to our network. We had hoped that we could train the network through ome ort of tochatic model, or through genetic programming, however we were unable to implement the training due to time contraint. We are providing the imulation here for future ue. Note: The imulation repreent each two bit weight in the hardware a an integer ranging from 0 to 3. The mot ignificant bit of the integer i the bit that i NANDed in hardware, the leat ignificant bit i the bit that i XORed. Thu for an input a: Weight Output ~a 3 A To create a network, ue: net = newbin(#input, layervector); where layervector i of the form [#input #input #input ] To create the network we defined in hardware, you call: net = newbin(5,[5 5 5]); To imulate, ue: reult = im(net, [in1_1 in1_2; in2_1 in 2_2; in3_1 in3_2; in4_1 in 4_2; in5_1 in5_2]); The value eparated by pace are different trial, the value eparated by emicolon are different input. Thi imulation i built in the neural net package in matlab. The weight for the firt layer are tored in the matrix net.iw{1}. The weight for ubequent layer are tored in net.lw{layer, layer+1}. Thi i becaue thee weight pecify the weight connecting that layer to the next layer. To ue thi imulation, place the following file in one directory and tart matlab from that directory (unix).

52 Smith & Maitra 52 File: ourweight.m function output = ourweight(weight, input) %produce an output uing our pecial little weight function if itr(weight) witch(weight) cae 'deriv' output = 'undefined'; otherwie error('unrecognized code.') end return end [wrow wcol] = ize(weight); [irow icol] = ize(input); for row=1:wrow for col = 1:icol oredweight = 0; for i=1:wcol witch(weight(row, i)) cae 0 weightedoutput = 1; cae 1; weightedoutput = 0; cae 2; weightedoutput = not(input(i, col)); cae 3; weightedoutput = input(i,col); otherwie mg = printf('invalid weight: %f', weight(row,i)) error(mg); end oredweight = oredweight weightedoutput; end output(row,col) = not(oredweight); end File:netnor.m function n = netnor(varargin) n = varargin{1}; if itr(n) witch n cae 'deriv', n = 'undefined'; otherwie error('unrecognized code.') end return end for i=2:length(varargin)

53 Smith & Maitra 53 n = n varargin(i); end %n = not(n); File: newbin.m function net = newbin(numinput, layer) %create a binary neural network %net = newbin(numinput, layer) % layer i a 1 by n matrix where each element i the ize of a layer numlayer = length(layer) if ia(layer,'cell') & (prod(ize(layer)) == length(layer)) layer = [layer{:}]; end %tructure net = network(1,numlayer); net.biaconnect = zero(numlayer,1); net.inputconnect(1,1) = 1; [j,i] = mehgrid(1:numlayer,1:numlayer); net.layerconnect = (j == (i-1)); net.outputconnect(numlayer) = 1; net.targetconnect(numlayer) = 1; %imulation net.input{1}.range = repmat([0 1], numinput,1); for i=1:numlayer net.layer{i}.ize = layer(i); net.layer{i}.tranferfcn = 'purelin'; end net.performfcn = 'me'; %Adaption %I dunno figure thi out later %training %neh %Initialization net.initfcn = 'initlay'; for i=1:numlayer net.layer{i}.initfcn = 'initwb'; net.layer{i}.netinputfcn = 'netnor'; end net.inputweight{1,1}.initfcn='initzero'; net.inputweight{1,1}.weightfcn='ourweight'; for i=2:numlayer net.layerweight{i,i-1}.initfcn='initzero'; net.layerweight{i,i-1}.weightfcn='ourweight'; end net = init(net);