Optimizing Image Classification: Automated Deep Learning Architecture Crafting with Network and Learning Hyperparameter Tuning

Abstract

This study introduces ETLBOCBL-CNN, an automated approach for optimizing convolutional neural network (CNN) architectures to address classification tasks of varying complexities. ETLBOCBL-CNN employs an effective encoding scheme to optimize network and learning hyperparameters, enabling the discovery of innovative CNN structures. To enhance the search process, it incorporates a competency-based learning concept inspired by mixed-ability classrooms during the teacher phase. This categorizes learners into competency-based groups, guiding each learner's search process by utilizing the knowledge of the predominant peers, the teacher solution, and the population mean. This approach fosters diversity within the population and promotes the discovery of innovative network architectures. During the learner phase, ETLBOCBL-CNN integrates a stochastic peer interaction scheme that encourages collaborative learning among learners, enhancing the optimization of CNN architectures. To preserve valuable network information and promote long-term population quality improvement, ETLBOCBL-CNN introduces a tri-criterion selection scheme that considers fitness, diversity, and learners' improvement rates. The performance of ETLBOCBL-CNN is evaluated on nine different image datasets and compared to state-of-the-art methods. Notably, ELTLBOCBL-CNN achieves outstanding accuracies on various datasets, including MNIST (99.72%), MNIST-RD (96.67%), MNIST-RB (98.28%), MNIST-BI (97.22%), MNST-RD + BI (83.45%), Rectangles (99.99%), Rectangles-I (97.41%), Convex (98.35%), and MNIST-Fashion (93.70%). These results highlight the remarkable classification accuracy of ETLBOCBL-CNN, underscoring its potential for advancing smart device infrastructure development.

Keywords: automatic network design; deep learning architecture; hyperparameter optimization; image classification; teaching–learning-based optimization.

Figure 1

The workflow of the original TLBO.

Figure 2

Typical architecture of sequential CNN.

Figure 3

Workflow of the ETLBOCBL-CNN framework.

Figure 4

Solution encoding scheme of ETLBOCBL-CNN to represent a potential CNN.

Figure 5

Decoding of network and learning hyperparameters for CNN construction.

Figure 6

Visual representation of calculating $\overline{X}.Mean$ in ETLBOCBL-CNN.

Figure 7

Visualization of the idea of competency-based learning introduced into the ETLOCBL-CNN’s modified teacher phase. Color dots refer to learners assigned to different groups.

Figure 8

Sample images of the datasets: (a) MNIST, (b) MNIST-RD, (c) MNIST-RB, (d) MNIST-BI, (e) MNIST-RD + BI, (f) Rectangles, (g) Rectangles-I, (h) Convex, and (i) Fashion.

Figure 8

Sample images of the datasets: (a) MNIST, (b) MNIST-RD, (c) MNIST-RB, (d) MNIST-BI, (e) MNIST-RD + BI, (f) Rectangles, (g) Rectangles-I, (h) Convex, and (i) Fashion.

Figure 9

Test errors obtained by ETLBOCBL-CNN while solving the eight datasets: (a) MNIST, (b) MNIST-RD, (c) MNIST-RB, (d) MNIST-BI, (e) MNIST-RD + BI, (f) Rectangles, (g) Rectangles-I, and (h) Convex.

Figure 9

Test errors obtained by ETLBOCBL-CNN while solving the eight datasets: (a) MNIST, (b) MNIST-RD, (c) MNIST-RB, (d) MNIST-BI, (e) MNIST-RD + BI, (f) Rectangles, (g) Rectangles-I, and (h) Convex.