bb6784ed3a731582ce5190cc38e45a4e electronics_a2022v11n10p1526.pdf 6c8e0b60d6b95ab0ad49564f32cf12e58c5cfd67 electronics_a2022v11n10p1526.pdf dad240360f7de26ad18bf6d02e8d6995e1bf69c4e4cc7816bc6cdd47a60c9357 electronics_a2022v11n10p1526.pdf Title: Ternary Neural Networks Based on on/off Memristors: Set-Up and Training Subject: Neuromorphic systems based on hardware neural networks (HNNs) are expected to be an energy and time-efficient computing architecture for solving complex tasks. In this paper, we consider the implementation of deep neural networks (DNNs) using crossbar arrays of memristors. More specifically, we considered the case where such devices can be configured in just two states: the low-resistance state (LRS) and the high-resistance state (HRS). HNNs suffer from several non-idealities that need to be solved when mapping our software-based models. A clear example in memristor-based neural networks is conductance variability, which is inherent to resistive switching devices, so achieving good performance in an HNN largely depends on the development of reliable weight storage or, alternatively, mitigation techniques against weight uncertainty. In this manuscript, we provide guidelines for a system-level designer where we take into account several issues related to the set-up of the HNN, such as what the appropriate conductance value in the LRS is or the adaptive conversion of current outputs at one stage to input voltages for the next stage. A second contribution is the training of the system, which is performed via offline learning, and considering the hardware imperfections, which in this case are conductance fluctuations. Finally, the resulting inference system is tested in two well-known databases from MNIST, showing that is competitive in terms of classification performance against the software-based counterpart. Additional advice and insights on system tuning and expected performance are given throughout the paper. Keywords: hardware neural networks; ternary networks; on/off memristors Author: Antoni Morell, Elvis Díaz Machado, Enrique Miranda, Guillem Boquet and Jose Lopez Vicario Creator: LaTeX with hyperref Producer: pdfTeX-1.40.21 CreationDate: Wed May 11 07:27:06 2022 CEST ModDate: Wed May 11 07:29:53 2022 CEST Tagged: no UserProperties: no Suspects: no Form: none JavaScript: no Pages: 17 Encrypted: no Page size: 595.276 x 841.89 pts (A4) Page rot: 0 File size: 1709669 bytes Optimized: no PDF version: 1.7 name type encoding emb sub uni object ID ------------------------------------ ----------------- ---------------- --- --- --- --------- CKBCVU+URWPalladioL-Bold Type 1 Custom yes yes no 10 0 EQWPLA+URWPalladioL-Roma Type 1 Custom yes yes no 15 0 SNZQTW+URWPalladioL-Ital Type 1 Custom yes yes no 19 0 MCBCCI+SourceSansRoman-Regular Type 1C Custom yes yes no 27 0 MCBCCM+SourceSansRoman.613wght Type 1C Custom yes yes no 31 0 MCBCCO+SourceSansRoman-Bold Type 1C Custom yes yes no 35 0 GKSXYT+TimesNewRomanPS-BoldItalicMT Type 1C WinAnsi yes yes no 47 0 TFIRTG+CMSY10 Type 1 Builtin yes yes no 69 0 SOSTRQ+CMR10 Type 1 Builtin yes yes no 73 0 HVSBKL+PazoMath Type 1 Builtin yes yes no 80 0 VKNWAU+URWPalladioL-BoldItal Type 1 Custom yes yes no 84 0 UYQZKL+PazoMath-Italic Type 1 Builtin yes yes no 91 0 AAAAAC+Courier TrueType MacRoman yes yes no 101 0 AAAABF+HelveticaNeue TrueType MacRoman yes yes no 104 0 AAAABG+Courier TrueType MacRoman yes yes no 107 0 AAAABL+Courier TrueType MacRoman yes yes no 110 0 AAAABP+Courier TrueType MacRoman yes yes no 113 0 AAAAAE+Courier TrueType MacRoman yes yes no 116 0 AAAAAH+Courier TrueType MacRoman yes yes no 119 0 AAAAAK+Courier TrueType MacRoman yes yes no 122 0 AAAAAN+Courier TrueType MacRoman yes yes no 125 0 AAAAAQ+Courier TrueType MacRoman yes yes no 128 0 AAAAAT+Courier TrueType MacRoman yes yes no 131 0 AAAAAX+Courier TrueType MacRoman yes yes no 134 0 AAAABA+Courier TrueType MacRoman yes yes no 137 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 140 0 ASBNUF+CMR7 Type 1 Custom yes yes no 145 0 DNQRZE+CMMIB10 Type 1 Custom yes yes no 150 0 ASBNUF+CMR7 Type 1 Custom yes yes no 155 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 157 0 ASBNUF+CMR7 Type 1 Custom yes yes no 159 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 161 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 166 0 ASBNUF+CMR7 Type 1 Custom yes yes no 168 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 170 0 ASBNUF+CMR7 Type 1 Custom yes yes no 172 0 SOSTRQ+CMR10 Type 1 Custom yes yes no 174 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 177 0 ICLVTH+CMR5 Type 1 Custom yes yes no 179 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 184 0 ASBNUF+CMR7 Type 1 Custom yes yes no 186 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 188 0 ICLVTH+CMR5 Type 1 Custom yes yes no 190 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 192 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 194 0 FYFSVN+CMMI5 Type 1 Custom yes yes no 196 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 201 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 203 0 ASBNUF+CMR7 Type 1 Custom yes yes no 205 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 207 0 ICLVTH+CMR5 Type 1 Custom yes yes no 209 0 SOSTRQ+CMR10 Type 1 Custom yes yes no 211 0 ZJYKVK+CMEX10 Type 1 Custom yes yes yes 213 0 DNQRZE+CMMIB10 Type 1 Custom yes yes no 219 0 ASBNUF+CMR7 Type 1 Custom yes yes no 221 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 223 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 225 0 DNQRZE+CMMIB10 Type 1 Custom yes yes no 227 0 ASBNUF+CMR7 Type 1 Custom yes yes no 229 0 DNQRZE+CMMIB10 Type 1 Custom yes yes no 231 0 AAAAAC+Courier TrueType MacRoman yes yes no 242 0 AAAAAE+Courier TrueType MacRoman yes yes no 245 0 AAAAAH+Courier TrueType MacRoman yes yes no 248 0 AAAAAK+Courier TrueType MacRoman yes yes no 251 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 254 0 SOSTRQ+CMR10 Type 1 Custom yes yes no 256 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 258 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 260 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 262 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 264 0 DNQRZE+CMMIB10 Type 1 Custom yes yes no 266 0 SOSTRQ+CMR10 Type 1 Custom yes yes no 268 0 SZMZGR+CMBX10 Type 1 Builtin yes yes no 279 0 HYYHEA+CMBSY10 Type 1 Builtin yes yes no 283 0 AAAAAC+Courier TrueType MacRoman yes yes no 295 0 AAAABF+Courier TrueType MacRoman yes yes no 298 0 AAAABJ+Courier TrueType MacRoman yes yes no 301 0 AAAABL+Courier TrueType MacRoman yes yes no 304 0 AAAABN+Courier TrueType MacRoman yes yes no 307 0 AAAABQ+Courier TrueType MacRoman yes yes no 310 0 AAAABT+Courier TrueType MacRoman yes yes no 313 0 AAAABY+Courier TrueType MacRoman yes yes no 316 0 AAAACC+Courier TrueType MacRoman yes yes no 319 0 AAAAAE+Courier TrueType MacRoman yes yes no 322 0 AAAAAH+Courier TrueType MacRoman yes yes no 325 0 AAAAAK+Courier TrueType MacRoman yes yes no 328 0 AAAAAN+Courier TrueType MacRoman yes yes no 331 0 AAAAAQ+Courier TrueType MacRoman yes yes no 334 0 AAAAAT+Courier TrueType MacRoman yes yes no 337 0 AAAAAX+Courier TrueType MacRoman yes yes no 340 0 AAAABA+Courier TrueType MacRoman yes yes no 343 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 346 0 ASBNUF+CMR7 Type 1 Custom yes yes no 348 0 DNQRZE+CMMIB10 Type 1 Custom yes yes no 350 0 ASBNUF+CMR7 Type 1 Custom yes yes no 352 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 354 0 ASBNUF+CMR7 Type 1 Custom yes yes no 356 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 358 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 360 0 ASBNUF+CMR7 Type 1 Custom yes yes no 362 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 364 0 ASBNUF+CMR7 Type 1 Custom yes yes no 366 0 SOSTRQ+CMR10 Type 1 Custom yes yes no 368 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 370 0 ICLVTH+CMR5 Type 1 Custom yes yes no 372 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 374 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 376 0 ASBNUF+CMR7 Type 1 Custom yes yes no 378 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 380 0 SOSTRQ+CMR10 Type 1 Custom yes yes no 382 0 TFIRTG+CMSY10 Type 1 Custom yes yes yes 384 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 388 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 390 0 ASBNUF+CMR7 Type 1 Custom yes yes no 392 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 394 0 ASBNUF+CMR7 Type 1 Custom yes yes no 396 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 398 0 ICLVTH+CMR5 Type 1 Custom yes yes no 400 0 DNQRZE+CMMIB10 Type 1 Custom yes yes no 402 0 ASBNUF+CMR7 Type 1 Custom yes yes no 404 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 406 0 DNQRZE+CMMIB10 Type 1 Custom yes yes no 408 0 ASBNUF+CMR7 Type 1 Custom yes yes no 410 0 ASBNUF+CMR7 Type 1 Custom yes yes no 412 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 414 0 NZOHYK+CMSY7 Type 1 Custom yes yes yes 416 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 422 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 424 0 DNQRZE+CMMIB10 Type 1 Custom yes yes no 426 0 ASBNUF+CMR7 Type 1 Custom yes yes no 428 0 DNQRZE+CMMIB10 Type 1 Custom yes yes no 430 0 ZJYKVK+CMEX10 Type 1 Builtin yes yes no 443 0 SYFPBV+CMMI10 Type 1 Builtin yes yes no 445 0 AAAAAB+Courier TrueType MacRoman yes yes no 459 0 AAAAAJ+Courier TrueType MacRoman yes yes no 462 0 AAAAAO+Courier TrueType MacRoman yes yes no 465 0 ASBNUF+CMR7 Type 1 Custom yes yes no 468 0 SOSTRQ+CMR10 Type 1 Custom yes yes no 470 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 472 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 474 0 ASBNUF+CMR7 Type 1 Custom yes yes no 476 0 SOSTRQ+CMR10 Type 1 Custom yes yes no 478 0 ZJYKVK+CMEX10 Type 1 Custom yes yes yes 480 0 DNQRZE+CMMIB10 Type 1 Custom yes yes no 483 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 485 0 NZOHYK+CMSY7 Type 1 Custom yes yes yes 487 0 ASBNUF+CMR7 Type 1 Custom yes yes no 490 0 SYFPBV+CMMI10 Type 1 Custom yes yes no 492 0 UGFVGL+CMMI7 Type 1 Custom yes yes no 494 0 ArialMT Type 3 Custom yes no no 505 0 DejaVuSans-Oblique Type 3 Custom yes no no 543 0 ArialMT Type 3 Custom yes no no 551 0 DejaVuSans-Oblique Type 3 Custom yes no no 586 0 GOFYPY+ArialMT Type 3 Custom yes yes no 598 0 GCWXDV+DejaVuSans-Oblique Type 3 Custom yes yes no 647 0 BMQQDV+DejaVuSans Type 3 Custom yes yes no 658 0 GOFYPY+ArialMT Type 3 Custom yes yes no 674 0 GCWXDV+DejaVuSans-Oblique Type 3 Custom yes yes no 719 0 BMQQDV+DejaVuSans Type 3 Custom yes yes no 734 0 GOFYPY+ArialMT Type 3 Custom yes yes no 755 0 GCWXDV+DejaVuSans-Oblique Type 3 Custom yes yes no 799 0 BMQQDV+DejaVuSans Type 3 Custom yes yes no 810 0 GOFYPY+ArialMT Type 3 Custom yes yes no 827 0 GCWXDV+DejaVuSans-Oblique Type 3 Custom yes yes no 869 0 BMQQDV+DejaVuSans Type 3 Custom yes yes no 884 0 Jhove (Rel. 1.22.1, 2019-04-17) Date: 2023-04-29 02:33:30 CEST RepresentationInformation: electronics_a2022v11n10p1526.pdf ReportingModule: PDF-hul, Rel. 1.12.1 (2019-04-17) LastModified: 2022-05-17 12:02:32 CEST Size: 1709669 Format: PDF Version: 1.7 Status: Well-Formed and valid SignatureMatches: PDF-hul MIMEtype: application/pdf PDFMetadata: Objects: 1051 FreeObjects: 1 IncrementalUpdates: 0 DocumentCatalog: PageLayout: SinglePage PageMode: UseNone Outlines: Item: Title: Introduction Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@44c8afef Item: Title: Scenario Description and Assumptions Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@7b69c6ba Item: Title: Proposed Algorithm Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@46daef40 Children: Item: Title: Training of Quantized Neural Networks Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@12f41634 Item: Title: Ternarization Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@13c27452 Item: Title: Adaptation of gH Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@262b2c86 Item: Title: Adaptation of aI2V Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@371a67ec Item: Title: Including Robustness in Perturbed Conductances Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@5ed828d Item: Title: Experimental Results Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@50d0686 Children: Item: Title: Adaptation of gH Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@7a3d45bd Item: Title: Adaptation of aI2V and Robustness to Perturbed Conductance Values Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@1e7c7811 Item: Title: Comparison with the Software-Based Neural Network Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@77ec78b9 Item: Title: Summary and Extension of Results Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@1a3869f4 Item: Title: Conclusions Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@a38d7a3 Item: Title: References Destination: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject@77f99a05 Info: Title: Ternary Neural Networks Based on on/off Memristors: Set-Up and Training Author: Antoni Morell, Elvis Díaz Machado, Enrique Miranda, Guillem Boquet and Jose Lopez Vicario Subject: Neuromorphic systems based on hardware neural networks (HNNs) are expected to be an energy and time-efficient computing architecture for solving complex tasks. In this paper, we consider the implementation of deep neural networks (DNNs) using crossbar arrays of memristors. More specifically, we considered the case where such devices can be configured in just two states: the low-resistance state (LRS) and the high-resistance state (HRS). HNNs suffer from several non-idealities that need to be solved when mapping our software-based models. A clear example in memristor-based neural networks is conductance variability, which is inherent to resistive switching devices, so achieving good performance in an HNN largely depends on the development of reliable weight storage or, alternatively, mitigation techniques against weight uncertainty. In this manuscript, we provide guidelines for a system-level designer where we take into account several issues related to the set-up of the HNN, such as what the appropriate conductance value in the LRS is or the adaptive conversion of current outputs at one stage to input voltages for the next stage. A second contribution is the training of the system, which is performed via offline learning, and considering the hardware imperfections, which in this case are conductance fluctuations. Finally, the resulting inference system is tested in two well-known databases from MNIST, showing that is competitive in terms of classification performance against the software-based counterpart. Additional advice and insights on system tuning and expected performance are given throughout the paper. Keywords: hardware neural networks; ternary networks; on/off memristors Creator: LaTeX with hyperref Producer: pdfTeX-1.40.21 CreationDate: Wed May 11 07:27:06 CEST 2022 ModDate: Wed May 11 07:29:53 CEST 2022 ID: 0xaa6b98af93e5b6103895282b42595aa2, 0xaa6b98af93e5b6103895282b42595aa2 Filters: FilterPipeline: FlateDecode Fonts: Type1: Font: BaseFont: HVSBKL+PazoMath FontSubset: true FirstChar: 68 LastChar: 229 FontDescriptor: FontName: HVSBKL+PazoMath Flags: Symbolic FontBBox: -40, -283, 878, 946 FontFile: true Font: BaseFont: SNZQTW+URWPalladioL-Ital FontSubset: true FirstChar: 45 LastChar: 122 FontDescriptor: FontName: SNZQTW+URWPalladioL-Ital Flags: Symbolic FontBBox: -170, -305, 1010, 941 FontFile: true EncodingDictionary: Differences: true Font: BaseFont: VKNWAU+URWPalladioL-BoldItal FontSubset: true FirstChar: 71 LastChar: 122 FontDescriptor: FontName: VKNWAU+URWPalladioL-BoldItal Flags: Symbolic FontBBox: -170, -300, 1073, 935 FontFile: true EncodingDictionary: Differences: true Font: BaseFont: TFIRTG+CMSY10 FontSubset: true FirstChar: 0 LastChar: 114 FontDescriptor: FontName: TFIRTG+CMSY10 Flags: Symbolic FontBBox: -29, -960, 1116, 775 FontFile: true Font: BaseFont: SZMZGR+CMBX10 FontSubset: true FirstChar: 43 LastChar: 43 FontDescriptor: FontName: SZMZGR+CMBX10 Flags: Symbolic FontBBox: -56, -250, 1164, 750 FontFile: true Font: BaseFont: SOSTRQ+CMR10 FontSubset: true FirstChar: 40 LastChar: 93 FontDescriptor: FontName: SOSTRQ+CMR10 Flags: Symbolic FontBBox: -40, -250, 1009, 750 FontFile: true Font: BaseFont: CKBCVU+URWPalladioL-Bold FontSubset: true FirstChar: 3 LastChar: 237 FontDescriptor: FontName: CKBCVU+URWPalladioL-Bold Flags: Symbolic FontBBox: -152, -301, 1000, 935 FontFile: true EncodingDictionary: Differences: true Font: BaseFont: UYQZKL+PazoMath-Italic FontSubset: true FirstChar: 97 LastChar: 182 FontDescriptor: FontName: UYQZKL+PazoMath-Italic Flags: Symbolic FontBBox: -70, -277, 902, 733 FontFile: true Font: BaseFont: HYYHEA+CMBSY10 FontSubset: true FirstChar: 0 LastChar: 0 FontDescriptor: FontName: HYYHEA+CMBSY10 Flags: Symbolic FontBBox: -26, -940, 1332, 825 FontFile: true Font: BaseFont: ZJYKVK+CMEX10 FontSubset: true FirstChar: 16 LastChar: 105 FontDescriptor: FontName: ZJYKVK+CMEX10 Flags: Symbolic FontBBox: -24, -2960, 1454, 772 FontFile: true Font: BaseFont: SYFPBV+CMMI10 FontSubset: true FirstChar: 60 LastChar: 62 FontDescriptor: FontName: SYFPBV+CMMI10 Flags: Symbolic FontBBox: -32, -250, 1048, 750 FontFile: true Font: BaseFont: EQWPLA+URWPalladioL-Roma FontSubset: true FirstChar: 2 LastChar: 243 FontDescriptor: FontName: EQWPLA+URWPalladioL-Roma Flags: Symbolic FontBBox: -166, -283, 1021, 943 FontFile: true EncodingDictionary: Differences: true Pages: Page: Sequence: 1 Annotations: Annotation: Subtype: Link Rect: 34, 319, 152, 328 Annotation: Subtype: Link Rect: 34, 309, 97, 316 Annotation: Subtype: Link Rect: 34, 159, 96, 181 Annotation: Subtype: Link Rect: 126, 86, 154, 95 Annotation: Subtype: Link Rect: 34, 74, 147, 83 Annotation: Subtype: Link Rect: 34, 756, 170, 792 Annotation: Subtype: Link Rect: 515, 756, 560, 792 Annotation: Subtype: Link Rect: 113, 656, 124, 670 Annotation: Subtype: Link Rect: 225, 656, 236, 670 Annotation: Subtype: Link Rect: 324, 656, 335, 670 Annotation: Subtype: Link Rect: 418, 656, 429, 670 Annotation: Subtype: Link Rect: 540, 656, 551, 670 Annotation: Subtype: Link Rect: 393, 214, 400, 223 ActionDest: 900 Annotation: Subtype: Link Rect: 504, 176, 511, 185 ActionDest: 900 Annotation: Subtype: Link Rect: 514, 176, 521, 185 ActionDest: 900 Annotation: Subtype: Link Rect: 210, 126, 217, 135 ActionDest: 900 Annotation: Subtype: Link Rect: 217, 126, 224, 135 ActionDest: 900 Annotation: Subtype: Link Rect: 225, 126, 232, 135 ActionDest: 900 Annotation: Subtype: Link Rect: 549, 101, 556, 110 ActionDest: 900 Annotation: Subtype: Link Rect: 124, 13, 286, 23 Annotation: Subtype: Link Rect: 399, 13, 560, 23 Annotation: Subtype: Link Rect: 34, 392, 86, 411 Page: Sequence: 2 Annotations: Annotation: Subtype: Link Rect: 191, 723, 198, 732 ActionDest: 903 Annotation: Subtype: Link Rect: 522, 685, 529, 694 ActionDest: 903 Annotation: Subtype: Link Rect: 241, 585, 253, 594 ActionDest: 903 Annotation: Subtype: Link Rect: 209, 572, 216, 581 ActionDest: 900 Annotation: Subtype: Link Rect: 244, 535, 250, 544 ActionDest: 900 Annotation: Subtype: Link Rect: 223, 484, 235, 493 ActionDest: 903 Annotation: Subtype: Link Rect: 237, 484, 249, 493 ActionDest: 903 Annotation: Subtype: Link Rect: 344, 472, 351, 481 ActionDest: 900 Annotation: Subtype: Link Rect: 425, 422, 437, 431 ActionDest: 903 Annotation: Subtype: Link Rect: 201, 396, 213, 406 ActionDest: 903 Annotation: Subtype: Link Rect: 298, 397, 310, 406 ActionDest: 903 Annotation: Subtype: Link Rect: 534, 283, 546, 293 ActionDest: 903 Annotation: Subtype: Link Rect: 332, 271, 339, 280 ActionDest: 900 Annotation: Subtype: Link Rect: 179, 183, 191, 192 ActionDest: 903 Annotation: Subtype: Link Rect: 370, 145, 381, 154 ActionDest: 903 Annotation: Subtype: Link Rect: 277, 108, 289, 117 ActionDest: 903 Annotation: Subtype: Link Rect: 491, 70, 503, 79 ActionDest: 903 Page: Sequence: 3 Annotations: Annotation: Subtype: Link Rect: 346, 660, 358, 669 ActionDest: 903 Annotation: Subtype: Link Rect: 392, 610, 404, 619 ActionDest: 903 Annotation: Subtype: Link Rect: 506, 607, 514, 619 ActionDest: 77 Annotation: Subtype: Link Rect: 298, 488, 305, 500 ActionDest: 77 Annotation: Subtype: Link Rect: 535, 476, 542, 488 ActionDest: 276 Annotation: Subtype: Link Rect: 385, 451, 391, 463 ActionDest: 496 Annotation: Subtype: Link Rect: 199, 438, 206, 450 ActionDest: 822 Annotation: Subtype: Link Rect: 348, 388, 355, 401 ActionDest: 88 Annotation: Subtype: Link Rect: 359, 335, 366, 351 ActionDest: 88 Annotation: Subtype: Link Rect: 279, 165, 286, 179 ActionDest: 88 Annotation: Subtype: Link Rect: 197, 102, 204, 114 ActionDest: 88 Page: Sequence: 4 Annotations: Annotation: Subtype: Link Rect: 325, 278, 337, 287 ActionDest: 903 Annotation: Subtype: Link Rect: 488, 217, 495, 229 ActionDest: 88 Page: Sequence: 5 Annotations: Annotation: Subtype: Link Rect: 213, 603, 225, 612 ActionDest: 903 Annotation: Subtype: Link Rect: 425, 541, 437, 549 ActionDest: 903 Annotation: Subtype: Link Rect: 281, 332, 288, 344 ActionDest: 276 Annotation: Subtype: Link Rect: 522, 147, 529, 159 ActionDest: 276 Page: Sequence: 6 Annotations: Annotation: Subtype: Link Rect: 544, 525, 556, 534 ActionDest: 903 Annotation: Subtype: Link Rect: 482, 134, 489, 146 ActionDest: 88 Annotation: Subtype: Link Rect: 439, 121, 446, 133 ActionDest: 276 Annotation: Subtype: Link Rect: 368, 73, 380, 82 ActionDest: 903 Page: Sequence: 7 Annotations: Annotation: Subtype: Link Rect: 358, 708, 365, 720 ActionDest: 440 Annotation: Subtype: Link Rect: 396, 571, 403, 591 ActionDest: 440 Annotation: Subtype: Link Rect: 179, 335, 191, 344 ActionDest: 903 Annotation: Subtype: Link Rect: 433, 142, 439, 154 ActionDest: 440 Page: Sequence: 8 Annotations: Annotation: Subtype: Link Rect: 329, 435, 341, 444 ActionDest: 903 Annotation: Subtype: Link Rect: 552, 160, 559, 172 ActionDest: 450 Annotation: Subtype: Link Rect: 511, 97, 523, 114 ActionDest: 447 Page: Sequence: 9 Annotations: Annotation: Subtype: Link Rect: 323, 720, 330, 732 ActionDest: 450 Annotation: Subtype: Link Rect: 547, 199, 554, 211 ActionDest: 440 Annotation: Subtype: Link Rect: 238, 118, 250, 127 ActionDest: 903 Page: Sequence: 10 Annotations: Annotation: Subtype: Link Rect: 292, 657, 303, 666 ActionDest: 903 Annotation: Subtype: Link Rect: 449, 657, 460, 666 ActionDest: 903 Annotation: Subtype: Link Rect: 470, 519, 482, 528 ActionDest: 903 Annotation: Subtype: Link Rect: 526, 504, 533, 516 ActionDest: 77 Annotation: Subtype: Link Rect: 216, 491, 223, 503 ActionDest: 496 Annotation: Subtype: Link Rect: 221, 469, 232, 478 ActionDest: 903 Annotation: Subtype: Link Rect: 233, 469, 245, 478 ActionDest: 903 Annotation: Subtype: Link Rect: 527, 293, 534, 305 ActionDest: 440 Annotation: Subtype: Link Rect: 261, 204, 268, 216 ActionDest: 499 Annotation: Subtype: Link Rect: 291, 204, 298, 216 ActionDest: 499 Annotation: Subtype: Link Rect: 236, 128, 243, 141 ActionDest: 440 Annotation: Subtype: Link Rect: 342, 128, 349, 141 ActionDest: 499 Annotation: Subtype: Link Rect: 507, 128, 514, 141 ActionDest: 499 Page: Sequence: 11 Annotations: Annotation: Subtype: Link Rect: 320, 128, 327, 140 ActionDest: 499 Page: Sequence: 12 Annotations: Annotation: Subtype: Link Rect: 218, 733, 225, 745 ActionDest: 499 Annotation: Subtype: Link Rect: 231, 569, 238, 581 ActionDest: 593 Page: Sequence: 13 Annotations: Annotation: Subtype: Link Rect: 229, 595, 236, 607 ActionDest: 669 Page: Sequence: 14 Annotations: Annotation: Subtype: Link Rect: 553, 691, 560, 704 ActionDest: 593 Annotation: Subtype: Link Rect: 376, 629, 382, 641 ActionDest: 750 Annotation: Subtype: Link Rect: 482, 579, 489, 591 ActionDest: 593 Annotation: Subtype: Link Rect: 374, 554, 381, 566 ActionDest: 593 Page: Sequence: 15 Annotations: Annotation: Subtype: Link Rect: 194, 632, 206, 644 ActionDest: 822 Annotation: Subtype: Link Rect: 360, 632, 367, 644 ActionDest: 669 Page: Sequence: 16 Annotations: Annotation: Subtype: Link Rect: 312, 509, 323, 519 ActionDest: 903 Annotation: Subtype: Link Rect: 327, 510, 338, 519 ActionDest: 903 Annotation: Subtype: Link Rect: 306, 270, 459, 281 Annotation: Subtype: Link Rect: 182, 258, 393, 269 Annotation: Subtype: Link Rect: 389, 180, 427, 191 Annotation: Subtype: Link Rect: 399, 157, 436, 168 Annotation: Subtype: Link Rect: 442, 157, 478, 168 Annotation: Subtype: Link Rect: 59, 135, 96, 145 Annotation: Subtype: Link Rect: 102, 135, 138, 145 Annotation: Subtype: Link Rect: 166, 112, 204, 122 Annotation: Subtype: Link Rect: 210, 112, 246, 122 Annotation: Subtype: Link Rect: 95, 89, 132, 99 Annotation: Subtype: Link Rect: 138, 89, 174, 99 Annotation: Subtype: Link Rect: 262, 65, 300, 76 Page: Sequence: 17 Annotations: Annotation: Subtype: Link Rect: 59, 699, 96, 710 Annotation: Subtype: Link Rect: 455, 676, 492, 687 Annotation: Subtype: Link Rect: 498, 676, 534, 687 Annotation: Subtype: Link Rect: 262, 652, 300, 664 Annotation: Subtype: Link Rect: 378, 618, 416, 629 Annotation: Subtype: Link Rect: 104, 584, 141, 594 Annotation: Subtype: Link Rect: 293, 549, 331, 560 Annotation: Subtype: Link Rect: 399, 526, 436, 537 Annotation: Subtype: Link Rect: 503, 491, 541, 502 Annotation: Subtype: Link Rect: 392, 481, 429, 491 Annotation: Subtype: Link Rect: 143, 445, 181, 456 Annotation: Subtype: Link Rect: 187, 445, 223, 456 Annotation: Subtype: Link Rect: 327, 422, 364, 433 Annotation: Subtype: Link Rect: 503, 365, 540, 376 Annotation: Subtype: Link Rect: 209, 320, 246, 330 Annotation: Subtype: Link Rect: 252, 320, 288, 330 Annotation: Subtype: Link Rect: 108, 285, 146, 295 Annotation: Subtype: Link Rect: 388, 238, 561, 249 Checksum: 5d273070 Type: CRC32 Checksum: bb6784ed3a731582ce5190cc38e45a4e Type: MD5 Checksum: 6c8e0b60d6b95ab0ad49564f32cf12e58c5cfd67 Type: SHA-1