9e0404477ff5648aecff234c248b8c3c  applsci-15-02481-v2.pdf
2f38e615fef309cee65098a0761ab6a0f7604350  applsci-15-02481-v2.pdf
3d822a73297424d0f703610bfcbcb2df5231970fe66126e087dc7cbf4c5c093d  applsci-15-02481-v2.pdf

Title:           Personalized Federated Learning with Progressive Local Training Strategy and Lightweight Classifier
Subject:         Data heterogeneity poses a significant challenge in federated learning (FL), which has become a central focus of contemporary research in artificial intelligence. Personalized federated learning (pFL), a specialized branch of FL, seeks to address this issue by tailoring models to the unique data distributions of individual clients. Despite its potential, current pFL frameworks face critical limitations, particularly in handling client training discontinuity. When clients are unable to engage in every training round, the resulting models tend to diverge from their local knowledge, leading to catastrophic forgetting. Moreover, existing frameworks often separate the model from the local classifier used for personalization, keeping the classifier local for extended periods. This inherent characteristic of classifiers frequently leads to overfitting on local training data, thereby impairing the generalization capability of the local models. To tackle these challenges, we propose a novel personalized federated learning framework, PFPS-LWC (Personalized Federated Learning with a Progressive Local Training Strategy and a Lightweight Classifier). Our approach introduces local knowledge recall and employs regularized classifiers to mitigate the effects of local knowledge forgetting and enhance the generalization of the models. We evaluated the performance of PFPS-LWC under varying degrees of data heterogeneity using the CIFAR10 and CIFAR100 datasets. Our method outperformed the state-of-the-art approach by up to 4.22% and consistently achieved the best performance across various heterogeneous environments, further demonstrating its effectiveness and robustness.
Keywords:        federatedlearning; personalized federated learning; data heterogeneity; catastrophic forgetting
Author:          Jianhao Liu, Wenjuan Gong, Ziyi Fang and Jordi Gonzàlez, Joel Rodrigues
Creator:         LaTeX with hyperref
Producer:        pdfTeX-1.40.25
CreationDate:    Wed Feb 26 08:32:13 2025 CET
ModDate:         Wed Feb 26 08:42:21 2025 CET
Custom Metadata: no
Metadata Stream: no
Tagged:          no
UserProperties:  no
Suspects:        no
Form:            none
JavaScript:      no
Pages:           26
Encrypted:       no
Page size:       595.276 x 841.89 pts (A4)
Page rot:        0
File size:       8330672 bytes
Optimized:       no
PDF version:     1.7


name                                 type              encoding         emb sub uni object ID
------------------------------------ ----------------- ---------------- --- --- --- ---------
FIEKKP+URWPalladioL-Roma             Type 1            Custom           yes yes yes     10  0
NPGVZT+URWPalladioL-Bold             Type 1            Custom           yes yes yes     16  0
LQKCFV+URWPalladioL-Ital             Type 1            Custom           yes yes yes     21  0
EVZBUN+CMSY10                        Type 1            Builtin          yes yes yes     55  0
UYLTRS+CMEX10                        Type 1            Builtin          yes yes yes     70  0
AZNIHD+PazoMath-Italic               Type 1            Builtin          yes yes yes     75  0
HVSBKL+PazoMath                      Type 1            Builtin          yes yes yes     80  0
SOSTRQ+CMR10                         Type 1            Builtin          yes yes yes     85  0
ORFKBL+EUFM10                        Type 1            Builtin          yes yes yes     96  0
DRQGWP+MSAM10                        Type 1            Builtin          yes yes yes    101  0
RAZVEJ+MSBM10                        Type 1            Builtin          yes yes yes    106  0
SYFPBV+CMMI10                        Type 1            Builtin          yes yes yes    114  0
HWTXSI+CMBSY10                       Type 1            Builtin          yes yes yes    134  0
IUOUHH+PazoMath-BoldItalic           Type 1            Builtin          yes yes yes    139  0
YLHTYT+URWPalladioL-BoldItal         Type 1            Custom           yes yes yes    144  0
GGZKKH+CMBX10                        Type 1            Builtin          yes yes yes    149  0


Jhove (Rel. 1.28.0, 2023-05-18)
 Date: 2025-04-03 02:10:04 CEST
 RepresentationInformation: applsci-15-02481-v2.pdf
  ReportingModule: PDF-hul, Rel. 1.12.4 (2023-03-16)
  LastModified: 2025-04-02 14:44:20 CEST
  Size: 8330672
  Format: PDF
  Version: 1.7
  Status: Well-Formed and valid
  SignatureMatches:
   PDF-hul
  MIMEtype: application/pdf
  PDFMetadata: 
   Objects: 383
   FreeObjects: 1
   IncrementalUpdates: 0
   DocumentCatalog: 
    PageLayout: SinglePage
    PageMode: UseNone
    Outlines: 
     Item: 
      Title: Introduction 
      Destination: section.1
     Item: 
      Title: Related Works
      Destination: section.2
      Children: 
       Item: 
        Title: Federated Learning Under Data Heterogeneity
        Destination: subsection.2.1
       Item: 
        Title: Personalized Federated Learning
        Destination: subsection.2.2
       Item: 
        Title: Comparison with Existing Methods
        Destination: subsection.2.3
     Item: 
      Title: Methodology
      Destination: section.3
      Children: 
       Item: 
        Title: Preliminaries
        Destination: subsection.3.1
       Item: 
        Title: Progressive Local Training Strategy
        Destination: subsection.3.2
       Item: 
        Title: Local Training with Lightweight Classifier
        Destination: subsection.3.3
       Item: 
        Title: The Convergence of PFPS-LWC
        Destination: subsection.3.4
       Item: 
        Title: An Overview of PFPS-LWC
        Destination: subsection.3.5
       Item: 
        Title: Discussion of the Proposed PFPS-LWC Method
        Destination: subsection.3.6
     Item: 
      Title: Experiments
      Destination: section.4
      Children: 
       Item: 
        Title: Experiment Setup
        Destination: subsection.4.1
       Item: 
        Title: A Comparison with State-of-the-Art Methods
        Destination: subsection.4.2
       Item: 
        Title: Ablation Experiments
        Destination: subsection.4.3
       Item: 
        Title: Hyperparameter Sensitivity Analysis
        Destination: subsection.4.4
       Item: 
        Title: Discussion of Other Parameters
        Destination: subsection.4.5
       Item: 
        Title: Analysis of Results and Future Research
        Destination: subsection.4.6
     Item: 
      Title: Conclusions
      Destination: section.5
     Item: 
      Title: Appendix A
      Destination: appendix.A.
     Item: 
      Title: References 
      Destination: appendix.B.
   Info: 
    Title: Personalized Federated Learning with Progressive Local Training Strategy and Lightweight Classifier
    Author: Jianhao Liu, Wenjuan Gong, Ziyi Fang and Jordi Gonzàlez, Joel Rodrigues
    Subject: Data heterogeneity poses a significant challenge in federated learning (FL), which has become a central focus of contemporary research in artificial intelligence. Personalized federated learning (pFL), a specialized branch of FL, seeks to address this issue by tailoring models to the unique data distributions of individual clients. Despite its potential, current pFL frameworks face critical limitations, particularly in handling client training discontinuity. When clients are unable to engage in every training round, the resulting models tend to diverge from their local knowledge, leading to catastrophic forgetting. Moreover, existing frameworks often separate the model from the local classifier used for personalization, keeping the classifier local for extended periods. This inherent characteristic of classifiers frequently leads to overfitting on local training data, thereby impairing the generalization capability of the local models. To tackle these challenges, we propose a novel personalized federated learning framework, PFPS-LWC (Personalized Federated Learning with a Progressive Local Training Strategy and a Lightweight Classifier). Our approach introduces local knowledge recall and employs regularized classifiers to mitigate the effects of local knowledge forgetting and enhance the generalization of the models. We evaluated the performance of PFPS-LWC under varying degrees of data heterogeneity using the CIFAR10 and CIFAR100 datasets. Our method outperformed the state-of-the-art approach by up to 4.22% and consistently achieved the best performance across various heterogeneous environments, further demonstrating its effectiveness and robustness.
    Keywords: federatedlearning; personalized federated learning; data heterogeneity; catastrophic forgetting
    Creator: LaTeX with hyperref
    Producer: pdfTeX-1.40.25
    CreationDate: Wed Feb 26 08:32:13 CET 2025
    ModDate: Wed Feb 26 08:42:21 CET 2025
   ID: 0x485cc7c2d43b9e768e1fb4fba1913918, 0x485cc7c2d43b9e768e1fb4fba1913918
   Filters: 
    FilterPipeline: FlateDecode
    FilterPipeline: DCTDecode
   Images: 
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 5683
      ImageHeight: 2835
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 5208
      ImageHeight: 1820
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 5415
      ImageHeight: 2752
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 2251
      ImageHeight: 1659
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 2251
      ImageHeight: 1662
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 3111
      ImageHeight: 2496
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 3810
      ImageHeight: 2487
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 3813
      ImageHeight: 2480
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 3118
      ImageHeight: 2484
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 3113
      ImageHeight: 2487
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 3804
      ImageHeight: 2473
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 3803
      ImageHeight: 2476
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 3118
      ImageHeight: 2489
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 5626
      ImageHeight: 2294
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 2926
      ImageHeight: 2412
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpg
      CompressionScheme: JPEG
      ImageWidth: 2942
      ImageHeight: 2386
      ColorSpace: RGB
      BitsPerSample: 8
      BitsPerSampleUnit: integer
   Fonts: 
    Type1: 
     Font: 
      BaseFont: ORFKBL+EUFM10
      FontSubset: true
      FirstChar: 82
      LastChar: 82
      FontDescriptor: 
       FontName: ORFKBL+EUFM10
       Flags: Symbolic
       FontBBox: -28, -257, 1055, 741
       FontFile: true
      ToUnicode: true
     Font: 
      BaseFont: DRQGWP+MSAM10
      FontSubset: true
      FirstChar: 54
      LastChar: 118
      FontDescriptor: 
       FontName: DRQGWP+MSAM10
       Flags: Symbolic
       FontBBox: 8, -463, 1331, 1003
       FontFile: true
      ToUnicode: true
     Font: 
      BaseFont: UYLTRS+CMEX10
      FontSubset: true
      FirstChar: 0
      LastChar: 118
      FontDescriptor: 
       FontName: UYLTRS+CMEX10
       Flags: Symbolic
       FontBBox: -24, -2960, 1454, 772
       FontFile: true
      ToUnicode: true
     Font: 
      BaseFont: HWTXSI+CMBSY10
      FontSubset: true
      FirstChar: 42
      LastChar: 42
      FontDescriptor: 
       FontName: HWTXSI+CMBSY10
       Flags: Symbolic
       FontBBox: -26, -940, 1332, 825
       FontFile: true
      ToUnicode: true
     Font: 
      BaseFont: FIEKKP+URWPalladioL-Roma
      FontSubset: true
      FirstChar: 2
      LastChar: 252
      FontDescriptor: 
       FontName: FIEKKP+URWPalladioL-Roma
       Flags: Symbolic
       FontBBox: -166, -283, 1021, 943
       FontFile: true
      EncodingDictionary: 
       Differences: true
      ToUnicode: true
     Font: 
      BaseFont: RAZVEJ+MSBM10
      FontSubset: true
      FirstChar: 69
      LastChar: 69
      FontDescriptor: 
       FontName: RAZVEJ+MSBM10
       Flags: Symbolic
       FontBBox: -55, -420, 2343, 920
       FontFile: true
      ToUnicode: true
     Font: 
      BaseFont: AZNIHD+PazoMath-Italic
      FontSubset: true
      FirstChar: 68
      LastChar: 115
      FontDescriptor: 
       FontName: AZNIHD+PazoMath-Italic
       Flags: Symbolic
       FontBBox: -70, -277, 902, 733
       FontFile: true
      ToUnicode: true
     Font: 
      BaseFont: IUOUHH+PazoMath-BoldItalic
      FontSubset: true
      FirstChar: 98
      LastChar: 98
      FontDescriptor: 
       FontName: IUOUHH+PazoMath-BoldItalic
       Flags: Symbolic
       FontBBox: -64, -272, 868, 740
       FontFile: true
      ToUnicode: true
     Font: 
      BaseFont: NPGVZT+URWPalladioL-Bold
      FontSubset: true
      FirstChar: 2
      LastChar: 224
      FontDescriptor: 
       FontName: NPGVZT+URWPalladioL-Bold
       Flags: Symbolic
       FontBBox: -152, -301, 1000, 935
       FontFile: true
      EncodingDictionary: 
       Differences: true
      ToUnicode: true
     Font: 
      BaseFont: HVSBKL+PazoMath
      FontSubset: true
      FirstChar: 68
      LastChar: 229
      FontDescriptor: 
       FontName: HVSBKL+PazoMath
       Flags: Symbolic
       FontBBox: -40, -283, 878, 946
       FontFile: true
      ToUnicode: true
     Font: 
      BaseFont: YLHTYT+URWPalladioL-BoldItal
      FontSubset: true
      FirstChar: 80
      LastChar: 80
      FontDescriptor: 
       FontName: YLHTYT+URWPalladioL-BoldItal
       Flags: Symbolic
       FontBBox: -170, -300, 1073, 935
       FontFile: true
      EncodingDictionary: 
       Differences: true
      ToUnicode: true
     Font: 
      BaseFont: SYFPBV+CMMI10
      FontSubset: true
      FirstChar: 60
      LastChar: 62
      FontDescriptor: 
       FontName: SYFPBV+CMMI10
       Flags: Symbolic
       FontBBox: -32, -250, 1048, 750
       FontFile: true
      ToUnicode: true
     Font: 
      BaseFont: LQKCFV+URWPalladioL-Ital
      FontSubset: true
      FirstChar: 2
      LastChar: 122
      FontDescriptor: 
       FontName: LQKCFV+URWPalladioL-Ital
       Flags: Symbolic
       FontBBox: -170, -305, 1010, 941
       FontFile: true
      EncodingDictionary: 
       Differences: true
      ToUnicode: true
     Font: 
      BaseFont: SOSTRQ+CMR10
      FontSubset: true
      FirstChar: 40
      LastChar: 93
      FontDescriptor: 
       FontName: SOSTRQ+CMR10
       Flags: Symbolic
       FontBBox: -40, -250, 1009, 750
       FontFile: true
      ToUnicode: true
     Font: 
      BaseFont: GGZKKH+CMBX10
      FontSubset: true
      FirstChar: 61
      LastChar: 61
      FontDescriptor: 
       FontName: GGZKKH+CMBX10
       Flags: Symbolic
       FontBBox: -56, -250, 1164, 750
       FontFile: true
      ToUnicode: true
     Font: 
      BaseFont: EVZBUN+CMSY10
      FontSubset: true
      FirstChar: 0
      LastChar: 114
      FontDescriptor: 
       FontName: EVZBUN+CMSY10
       Flags: Symbolic
       FontBBox: -29, -960, 1116, 775
       FontFile: true
      ToUnicode: true
   Pages: 
    Page: 
     Sequence: 1
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 81, 177, 152, 186
      Annotation: 
       Subtype: Link
       Rect: 34, 165, 103, 174
      Annotation: 
       Subtype: Link
       Rect: 36, 75, 152, 84
      Annotation: 
       Subtype: Link
       Rect: 34, 63, 89, 72
      Annotation: 
       Subtype: Link
       Rect: 34, 773, 193, 809
      Annotation: 
       Subtype: Link
       Rect: 515, 773, 560, 809
      Annotation: 
       Subtype: Link
       Rect: 192, 663, 203, 677
      Annotation: 
       Subtype: Link
       Rect: 336, 663, 347, 677
      Annotation: 
       Subtype: Link
       Rect: 441, 663, 452, 677
      Annotation: 
       Subtype: Link
       Rect: 431, 165, 438, 174
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 441, 165, 448, 174
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 494, 165, 501, 174
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 501, 165, 508, 174
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 203, 151, 210, 160
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 211, 151, 218, 160
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 348, 151, 355, 160
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 358, 151, 370, 160
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 404, 109, 416, 118
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 419, 109, 431, 118
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 422, 27, 560, 37
      Annotation: 
       Subtype: Link
       Rect: 34, 332, 87, 351
    Page: 
     Sequence: 2
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 304, 744, 316, 754
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 244, 730, 256, 740
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 259, 730, 271, 740
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 407, 661, 419, 670
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 328, 647, 340, 656
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 544, 647, 556, 656
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 261, 633, 273, 642
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 338, 619, 350, 628
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 409, 619, 421, 628
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 261, 577, 273, 586
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 273, 577, 285, 586
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 440, 577, 452, 586
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 373, 491, 380, 503
       ActionDest: 44
      Annotation: 
       Subtype: Link
       Rect: 468, 113, 475, 125
       ActionDest: 48
    Page: 
     Sequence: 3
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 447, 319, 453, 331
       ActionDest: 48
      Annotation: 
       Subtype: Link
       Rect: 451, 305, 458, 317
       ActionDest: 67
      Annotation: 
       Subtype: Link
       Rect: 344, 292, 351, 304
       ActionDest: 123
      Annotation: 
       Subtype: Link
       Rect: 455, 278, 462, 290
       ActionDest: 180
    Page: 
     Sequence: 4
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 470, 616, 477, 628
       ActionDest: 52
      Annotation: 
       Subtype: Link
       Rect: 432, 177, 444, 186
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 346, 122, 358, 131
       ActionDest: 193
    Page: 
     Sequence: 5
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 414, 717, 426, 726
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 468, 717, 480, 726
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 543, 717, 555, 726
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 219, 703, 231, 712
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 261, 703, 273, 712
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 545, 647, 557, 656
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 232, 633, 244, 642
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 243, 563, 255, 572
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 321, 563, 333, 572
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 398, 563, 410, 572
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 318, 230, 330, 240
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 496, 175, 508, 184
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 543, 175, 555, 184
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 222, 161, 234, 170
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 343, 119, 355, 128
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 422, 119, 434, 128
       ActionDest: 196
    Page: 
     Sequence: 6
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 214, 759, 226, 767
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 407, 717, 419, 726
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 190, 661, 202, 670
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 525, 619, 537, 628
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 204, 325, 211, 337
       ActionDest: 64
      Annotation: 
       Subtype: Link
       Rect: 70, 269, 79, 276
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 71, 261, 80, 268
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 69, 253, 78, 260
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 71, 245, 80, 252
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 69, 237, 78, 244
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 72, 230, 81, 236
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 353, 164, 365, 173
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 355, 150, 367, 159
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 201, 122, 213, 131
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 261, 122, 273, 131
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 333, 122, 345, 131
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 211, 80, 223, 89
       ActionDest: 196
    Page: 
     Sequence: 7
    Page: 
     Sequence: 8
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 218, 75, 230, 84
       ActionDest: 199
    Page: 
     Sequence: 9
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 336, 585, 343, 597
       ActionDest: 93
      Annotation: 
       Subtype: Link
       Rect: 413, 532, 425, 541
       ActionDest: 199
      Annotation: 
       Subtype: Link
       Rect: 277, 462, 289, 471
       ActionDest: 199
    Page: 
     Sequence: 10
    Page: 
     Sequence: 11
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 409, 350, 419, 362
       ActionDest: 183
      Annotation: 
       Subtype: Link
       Rect: 415, 296, 422, 308
       ActionDest: 123
    Page: 
     Sequence: 12
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 227, 102, 239, 111
       ActionDest: 199
    Page: 
     Sequence: 13
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 232, 759, 244, 767
       ActionDest: 199
      Annotation: 
       Subtype: Link
       Rect: 471, 502, 478, 514
       ActionDest: 126
      Annotation: 
       Subtype: Link
       Rect: 543, 192, 555, 201
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 202, 178, 214, 187
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 263, 178, 275, 187
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 345, 178, 357, 187
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 545, 150, 556, 159
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 227, 136, 239, 145
       ActionDest: 193
    Page: 
     Sequence: 14
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 279, 437, 286, 449
       ActionDest: 131
      Annotation: 
       Subtype: Link
       Rect: 289, 437, 296, 449
       ActionDest: 154
      Annotation: 
       Subtype: Link
       Rect: 488, 395, 495, 407
       ActionDest: 131
      Annotation: 
       Subtype: Link
       Rect: 283, 325, 290, 337
       ActionDest: 154
      Annotation: 
       Subtype: Link
       Rect: 206, 203, 217, 211
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 207, 193, 218, 201
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 205, 184, 216, 191
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 207, 174, 218, 182
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 205, 165, 215, 172
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 209, 155, 219, 163
       ActionDest: 196
    Page: 
     Sequence: 15
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 206, 687, 217, 695
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 207, 677, 218, 685
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 205, 668, 216, 675
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 207, 658, 218, 666
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 205, 649, 215, 656
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 209, 639, 219, 647
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 222, 499, 236, 508
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 223, 486, 237, 495
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 220, 472, 234, 482
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 223, 459, 237, 469
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 220, 446, 234, 455
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 225, 433, 239, 442
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 223, 370, 230, 382
       ActionDest: 154
      Annotation: 
       Subtype: Link
       Rect: 334, 286, 341, 298
       ActionDest: 131
      Annotation: 
       Subtype: Link
       Rect: 344, 286, 351, 298
       ActionDest: 154
      Annotation: 
       Subtype: Link
       Rect: 544, 108, 556, 117
       ActionDest: 196
      Annotation: 
       Subtype: Link
       Rect: 226, 94, 238, 103
       ActionDest: 193
      Annotation: 
       Subtype: Link
       Rect: 424, 77, 431, 89
       ActionDest: 154
    Page: 
     Sequence: 16
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 247, 672, 254, 684
       ActionDest: 157
      Annotation: 
       Subtype: Link
       Rect: 274, 672, 281, 684
       ActionDest: 164
    Page: 
     Sequence: 17
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 474, 299, 481, 311
       ActionDest: 164
    Page: 
     Sequence: 18
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 292, 451, 299, 463
       ActionDest: 171
    Page: 
     Sequence: 19
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 474, 558, 481, 570
       ActionDest: 175
      Annotation: 
       Subtype: Link
       Rect: 208, 229, 215, 241
       ActionDest: 175
    Page: 
     Sequence: 20
    Page: 
     Sequence: 21
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 228, 369, 418, 380
      Annotation: 
       Subtype: Link
       Rect: 232, 356, 417, 367
    Page: 
     Sequence: 22
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 456, 412, 471, 424
       ActionDest: 183
      Annotation: 
       Subtype: Link
       Rect: 340, 253, 355, 266
       ActionDest: 187
      Annotation: 
       Subtype: Link
       Rect: 381, 253, 396, 266
       ActionDest: 187
      Annotation: 
       Subtype: Link
       Rect: 262, 125, 282, 137
       ActionDest: 187
      Annotation: 
       Subtype: Link
       Rect: 291, 125, 311, 137
       ActionDest: 187
    Page: 
     Sequence: 23
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 344, 639, 359, 651
       ActionDest: 187
      Annotation: 
       Subtype: Link
       Rect: 347, 625, 362, 637
       ActionDest: 187
      Annotation: 
       Subtype: Link
       Rect: 344, 523, 359, 535
       ActionDest: 187
      Annotation: 
       Subtype: Link
       Rect: 467, 509, 481, 521
       ActionDest: 187
      Annotation: 
       Subtype: Link
       Rect: 212, 495, 227, 507
       ActionDest: 187
    Page: 
     Sequence: 24
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 361, 555, 399, 566
      Annotation: 
       Subtype: Link
       Rect: 306, 530, 344, 541
      Annotation: 
       Subtype: Link
       Rect: 430, 504, 468, 515
      Annotation: 
       Subtype: Link
       Rect: 183, 480, 220, 490
      Annotation: 
       Subtype: Link
       Rect: 227, 480, 263, 490
      Annotation: 
       Subtype: Link
       Rect: 458, 377, 496, 388
      Annotation: 
       Subtype: Link
       Rect: 378, 351, 415, 362
      Annotation: 
       Subtype: Link
       Rect: 180, 287, 217, 298
      Annotation: 
       Subtype: Link
       Rect: 199, 262, 236, 273
      Annotation: 
       Subtype: Link
       Rect: 196, 96, 233, 107
    Page: 
     Sequence: 25
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 113, 744, 151, 754
      Annotation: 
       Subtype: Link
       Rect: 248, 692, 285, 703
      Annotation: 
       Subtype: Link
       Rect: 259, 679, 296, 690
      Annotation: 
       Subtype: Link
       Rect: 257, 590, 295, 601
      Annotation: 
       Subtype: Link
       Rect: 59, 259, 96, 269
    Page: 
     Sequence: 26
     Annotations: 
      Annotation: 
       Subtype: Link
       Rect: 294, 756, 560, 767
      Annotation: 
       Subtype: Link
       Rect: 56, 743, 160, 754
  Checksum: ed5a1ffb
   Type: CRC32
  Checksum: 9e0404477ff5648aecff234c248b8c3c
   Type: MD5
  Checksum: 2f38e615fef309cee65098a0761ab6a0f7604350
   Type: SHA-1
  Checksum: 3d822a73297424d0f703610bfcbcb2df5231970fe66126e087dc7cbf4c5c093d
   Type: SHA-256