Cover Image: Five different types of dogs. This woodcut image from 1547, created by an unknown artist is part of the Wellcome Collection. Modern Convolutional Neural Networks (CNNs) could analyze and classify these dogs by digitizing the image, training with labeled dog breed datasets, and identifying the breeds using layers that extract features, reduce dimensionality, and classify the image. Image Source: Wellcome Collection (Museum and library in London, United Kingdom) CC BY 4.0. Source: Wikimedia Commons.

Prof. Maurício Veloso Brant Pinheiro

This article is based on my class notes from the Special Topics D: Artificial Intelligence (TIA) course, a 60-hour program for undergraduate and graduate students at the Physics Department of UFMG.

Abstract: Convolutional Neural Networks (CNNs) have revolutionized computer vision, transforming how machines perceive visual data. Inspired by the human visual system, CNNs are essential for tasks such as image, sound, and video recognition, as well as natural language processing. This article explores CNNs from their historical development to their architecture, using an example of recognizing Xs and Os in a 9×9 pixel matrix. We cover convolutional layers, kernels, pooling layers, and activation functions, and include a practical application. The implementation is demonstrated using a CNN with Python and TensorFlow on Google Colab, training it on the MNIST database for handwritten digit recognition. We conclude by discussing other applications of CNNs, such as medical image analysis and financial data processing, highlighting their versatility.

1. Introduction

Convolutional Neural Networks (CNNs) have brought about a paradigm shift in the realm of computer vision, revolutionizing how machines perceive and interpret visual data. Inspired by the intricate workings of the human visual cortex, CNNs excel at recognizing patterns and structures in visual data, making them indispensable in tasks such as image, sound, and video recognition, image classification, and even natural language processing, where spatial data representations play a crucial role.

In this paper, we embark on a journey into the realm of Convolutional Neural Networks (CNNs). We commence our exploration with a concise overview of their historical development, tracing the evolution of CNNs from their foundational roots to their current state of prominence. Following this historical backdrop, we delve into a captivating comparison between the human visual system and the sophisticated mechanisms through which CNNs mirror its processing dynamics. By discerning the striking parallels between biological vision and machine vision, we unlock profound insights into the remarkable efficacy of CNNs in decoding and interpreting visual data.

Moving forward, we delve into the basic architecture of CNNs using a simple yet illustrative example: recognizing patterns in a 9×9 pixel matrix representing Xs and Os. Through this example, we elucidate the fundamental concepts underlying CNNs, such as convolutional layers, kernels, pooling layers, and activation functions. To aid in visualizing the matrix operations involved, we provide a sandbox for matrix plotting, enabling readers to interactively explore the convolutional process.

Next, we transition to a more practical demonstration by demonstrating how to implement a full-fledged CNN in Python, using TensorFlow, a popular deep learning framework, within the Google Colab environment. We train the CNN using the MNIST database, a seminal example in the field of CNNs, for handwritten digit recognition. By showcasing the training process and evaluating the model’s performance, we illustrate the real-world applicability of CNNs in image classification tasks.

Finally, we broaden the scope of our discussion to explore applications of CNNs beyond image recognition, highlighting their versatility in analyzing structured data beyond traditional visual domains. From analyzing medical images to processing financial data, CNNs offer a powerful toolkit for extracting meaningful insights from diverse datasets, underscoring their significance in contemporary data-driven applications.

Through this comprehensive exploration, we aim to provide readers with a deeper understanding of CNNs, from their theoretical underpinnings to practical applications, empowering them to harness the transformative potential of this groundbreaking technology in their own endeavors.

---

2. A Brief History of the CNNs

The concept of convolutional neural networks was first introduced by Yann LeCun and his collaborators in the late 1980s and early 1990s. Their work on the LeNet architecture demonstrated the potential of CNNs for digit recognition tasks, such as reading zip codes and recognizing handwritten digits.

The field took a significant leap forward with the introduction of the AlexNet architecture by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton in 2012. AlexNet competed in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) and achieved a top-5 error rate of 15.3%, significantly outperforming the runner-up with an error rate of 26.2%. This marked a turning point in the field, showcasing the power of deep learning and CNNs for image classification at scale.

Following AlexNet, several other architectures have pushed the boundaries of CNN performance. Notable among these are the VGGNet by the Visual Geometry Group at Oxford, which emphasized depth through very small (3×3) convolution filters, and Google’s Inception network (GoogLeNet), which introduced the idea of inception modules to efficiently capture spatial hierarchies.

More recently, the ResNet architecture, developed by Kaiming He and colleagues, introduced the concept of residual learning, allowing for the training of even deeper networks by addressing the vanishing gradient problem. ResNet’s performance in the ILSVRC 2015 competition set a new standard with a top-5 error rate of 3.57%.

These breakthroughs have not only established CNNs as the leading approach for a wide range of computer vision tasks but have also paved the way for further innovations in deep learning.

---

3. Human Vision and CNNs

To comprehend how Convolutional Neural Networks (CNNs) emulate the human visual system, it’s essential to delve into the intricate process of image formation and brain processing. Our visual experience commences with light entering our eyes, journeying through the cornea, pupil, and lens, ultimately converging onto the retina at the eye’s rear. Within the retina, specialized photoreceptor cells, rods, and cones, convert light stimuli into electrical signals, initiating the journey of visual perception relayed through the optic nerve to various brain regions dedicated to visual processing.

As light-encoded electrical signals traverse the optic nerve, they reach the brain’s visual cortex, situated at the rear of the brain. Within the visual cortex, visual information undergoes hierarchical processing, beginning with rudimentary features like edges and textures identified in early layers. Subsequent layers intricately weave together these elementary features, gradually unraveling complex patterns encompassing shapes, objects, and scenes. This hierarchical approach enables the brain to construct a nuanced representation of the visual landscape, enriching our understanding.

Operating alongside hierarchical processing, the visual cortex engages in parallel processing, facilitating simultaneous analysis of diverse visual aspects. Through parallel streams, the cortex adeptly juggles myriad visual cues, enhancing the brain’s efficiency in decoding and comprehending visual input. One stream may focus on identifying objects (ventral stream), while another processes motion or spatial relationships (dorsal stream). This parallel processing mirrors the various streams of visual information processed by different regions of the visual cortex.

Now, let’s relate this to CNNs. Inspired by the human visual cortex, CNNs replicate its hierarchical and parallel processing mechanisms. Convolutional layers meticulously analyze local image regions, detecting features akin to receptive fields in the visual cortex. Subsequent layers abstract higher-level features, mirroring hierarchical processing. Moreover, CNN architectures often incorporate parallel processing, with multiple branches handling different aspects of input images, akin to the visual cortex’s parallel streams.

By mimicking these principles, CNNs excel at recognizing patterns in visual data, particularly in image recognition tasks. They learn to identify unique features through convolutional layers, pooling layers summarize detected features, and activation functions introduce non-linearity. This hierarchical feature extraction process parallels the human visual system, making CNNs highly effective in computer vision applications.

---

4. The CNN Basics: Recognizing Xs and Os

Let’s take a look at a simple problem to understand the architecture of a Convolutional Neural Network (CNN). We are given a dataset of 9×9 pixel images, where each image contains either an “X” or an “O”. Our task is to build a Convolutional Neural Network (CNN) that can accurately classify these images.

However, the machine encounters several challenges due to variations in the appearance of these figures. Modifications such as changes in scale, rotations, translations, and color variations can make it difficult for the machine to match the expected “X” or “O” with the input images.

What seems like a simple problem for the human brain—recognizing an “X” or “O” despite these variations—is actually a very difficult problem for a machine. Our brains are remarkably adept at recognizing patterns and shapes, even when they are transformed in various ways. In contrast, a machine must be explicitly trained to handle these variations.

Fortunately, CNNs, by mimicking the architecture of the human visual cortex, can learn to separate and recognize features within images despite these transformations. Here’s how a CNN would approach solving this problem step-by-step:

We begin with the convolution stage, which employs filters to identify patterns within the image. In the context of CNNs, a filter, also referred to as a kernel, is a small matrix of weights that traverses the input data (such as an image) to generate a feature map. These filters are designed to capture distinct features of the input data, such as edges, textures, or other patterns.

The feature map is built by multiplying each value of the filter with the corresponding value in the input matrix overlaid by the filterand then sliding the filter across the input matrix to build the entire feature map.

Care must be taken with the borders of the input matrix. In our case, we will use three 3×3 filters that represent the single diagonals and a reduced X pattern. The kernels are initially overlaid on the input image so that their borders coincide at the top left corner.

After the multiplication and averaging steps, the feature map is constructed.

The output of the convolution is a feature map for each kernel. In our case, the feature maps are 7×7 matrices due to border handling. When no padding is applied and the stride is set to 1, the dimensions of the output are reduced by a factor of 2. Padding involves adding extra pixels around the input image to maintain its original dimensions after convolutional operations, while the stride determines the filter’s movement per step during convolution.

After convolving the input matrix with the kernels, one can apply either normalization using ReLU (Rectified Linear Unit) or max-pooling to reduce the dimensionality of the feature maps. A ReLU activation function is applied to the feature maps setting all negative values to zero.

The 3×3 max pooling operation can also be applied to each feature map, which reduces the dimensionality by taking the maximum value from each 3×3 region (and the left border).

The feature maps are then stacked up and become the new input matrices for a new convolution layer.

The process is then repeated adding more convolution layers with normalization and pooling layers.

The 3 feature maps of size 4×4 are then flattened into a single vector of length 12. This vector represents the extracted features from the original 9×9 image.

Finally, the flattened vector is passed through a fully connected dense layer (or layers) with the appropriate number of input neurons. Let’s assume we have 2 neurons in the output layer, corresponding to the two classes: “X” and “O”.

At this point you should try our app Matrix_Plotter

For the transition between the two dense layers a Softmax activation function is applied to the output neurons to convert their values into probabilities. The Softmax activation function is commonly used in the output layer of a neural network for classification tasks. It converts the raw output values of the neurons into probabilities, making it easier to interpret the network’s predictions. This allows us to interpret the network’s output as the likelihood of each class given the input data.

After feeding-foward the dense layer, the output layer neurons provides probabilities for each class. For instance, it might output [0.63, 0.37], indicating an 63 % probability that the image contains an “X” and a 37% probability that it contains an “O” (for the particular feature vector above).

Thus we have the complete architeture of this CNN.

The next step is to train it with labeled input data with as much variations possible (rotations, translations, scaling and fadding colors). This is done by backpropagation using otimizers like simple Stochastic Gradient Descent.

Once trained, the model must undergo testing and validation to assess its performance accurately. This involves selecting, hyperparameters, suitable sizes for mini-batches and the testing and validation datasets. Once these are appropriately chosen, the CNN model is ready for making predictions on new data. It’s important to note that the kernels, analogous to the weights and biases in a standard neural network, are optimized by the network during the training process. Typically, these kernels are initialized randomly, and through the iterative optimization process in the training, they evolve to capture meaningful features from the input data.

---

5. Programing a simple CNN to recognize handwritten digits

Now that you’ve grasped the fundamentals, it’s time to dive into building your first CNN. The code we are about to implement performs image recognition and classification on numbers from the MNIST database. If you’re solely interested in the outcomes and lack coding expertise, fret not, for GPT-4 will handle the coding for you, allowing you to skip the technicalities and go directly to the results.

However, if you possess a basic understanding of Python and TensorFlow, you have the opportunity to enhance the code further. Once the code is generated, it’s a breeze to copy it into a new notebook on Google Colab and execute it in the Cloud. Simple, right? Let’s demonstrate how this can be achieved in a matter of seconds.

At GPT-4o we input a prompt like, for example:

Prompt> Write an example of a CNN Python code that runs in Google Colab for handwritten digit identification using TensorFlow, Keras, Pandas, and the MNIST database.

As a result, we obtain the following answer along with the complete code.

A simple and effective example to demonstrate Convolutional Neural Networks (CNNs) is the classification of the MNIST dataset of handwritten digits. This dataset is widely used in teaching because of its simplicity and the clear, visual results it produces.

Below is the complete commented code generated from this single prompt. Click here to view the code on GitHub.

The Code:

The first segment of this code begins by importing the necessary libraries for building and training a neural network using the MNIST dataset. The libraries imported are TensorFlow, Keras, NumPy, and Matplotlib. TensorFlow is a popular machine learning library developed by Google, and Keras is an API designed to work on top of TensorFlow to facilitate the creation of neural networks. NumPy is used for numerical operations, particularly for handling arrays, and Matplotlib is a plotting library used for visualizing data.

Next, the MNIST dataset is loaded. MNIST is a widely used dataset consisting of 70,000 grayscale images of handwritten digits (0-9), with 60,000 images designated for training and 10,000 for testing. The images are 28×28 pixels in size, and the pixel values range from 0 to 255.

The images are then reshaped and normalized. The reshaping adds a single channel dimension to the images, making their shape (28, 28, 1), which is necessary for certain neural network layers. Normalizing the images scales the pixel values to the range [0, 1], which can help improve the training process by ensuring the inputs are in a consistent range.

Finally, the labels, which are categorical integers representing the digits, are one-hot encoded. One-hot encoding transforms the labels into binary vectors, where the index of the digit is marked with a 1, and all other indices are 0. This encoding is necessary for the neural network to properly interpret the labels during training.

In this second segment of the code, a Convolutional Neural Network (CNN) model is built, compiled, trained, and evaluated using the MNIST dataset.

First, the model is defined using Keras’ Sequential class, which allows for a linear stack of layers. The CNN model starts with a convolutional layer (Conv2D) that applies 32 filters with a 3×3 kernel size, followed by a ReLU activation function. The input_shape is specified as (28, 28, 1) to match the shape of the MNIST images. This is followed by a max-pooling layer (MaxPooling2D) that reduces the spatial dimensions by taking the maximum value over a 2×2 window.

Another convolutional layer with 64 filters and a 3×3 kernel, followed by another max-pooling layer, is added. This pattern is repeated once more with an additional convolutional layer with 64 filters. After the convolutional layers, the model is flattened, converting the 2D matrix data into a 1D vector, which is then fed into a densely connected layer (Dense) with 64 units and a ReLU activation. The final layer is a dense layer with 10 units (one for each digit) and a softmax activation function, which outputs a probability distribution over the 10 classes.

The model is then compiled, specifying the Adam optimizer, categorical cross-entropy as the loss function, and accuracy as the metric to track.

Training the model is performed using the fit method, which takes in the training images and labels, runs for 10 epochs, and uses the test set for validation after each epoch.

Finally, the model is evaluated using the evaluate method, which returns the test loss and accuracy. The test accuracy is printed to give an indication of the model’s performance on the unseen test data.

The following part of the code defines a function plot_history that visualizes the training and validation accuracy and loss of a machine learning model. The function takes a history object as input, which is typically returned by model training processes in deep learning frameworks such as Keras. The plt.figure function is used to create a new figure with a specified size, while plt.subplot is used to define a grid of subplots within the figure.

In the first subplot, the training and validation accuracy values are plotted against the epochs. The training accuracy is labeled as ‘Train Accuracy’ and plotted in one color, while the validation accuracy is labeled as ‘Test Accuracy’ and plotted in another color. The plot is given a title ‘Model Accuracy’, and the x and y axes are labeled ‘Epoch’ and ‘Accuracy’, respectively. A legend is placed in the lower right corner to distinguish between the two lines.

In the second subplot, the training and validation loss values are plotted similarly. The training loss is labeled ‘Train Loss’ and the validation loss is labeled ‘Test Loss’. The plot is titled ‘Model Loss’, and the axes are labeled ‘Epoch’ and ‘Loss’. The legend for this plot is placed in the upper right corner. Finally, plt.tight_layout() is called to ensure there is no overlap between subplots, and plt.show() displays the plots. After defining the function, it is called with the history object to generate the plots. This visualization helps in understanding the model’s performance over training epochs by comparing accuracy and loss for both training and validation datasets.

This another part of the code aims to visualize the predictions of a machine learning model on a set of test images. The first line executes the model’s prediction function on the test images, storing the predicted outputs in the predictions variable.

Following this, a function plot_image is defined to plot an individual image along with its predicted and true labels. This function takes four parameters: the index i, the array of predictions predictions_array, the true label true_label, and the image data img. Within the function, several plotting customizations are applied: the grid is disabled, and the x and y ticks are removed to enhance visual clarity. The image is displayed using plt.imshow, with a binary color map to represent the grayscale image.

The function then determines the predicted label by finding the index of the maximum value in predictions_array, and the true label by finding the index of the maximum value in true_label. It uses these labels to set the text color for the x-axis label: blue if the prediction is correct, and red if it is incorrect. Finally, the x-axis label is set to display both the predicted and true labels in the specified color.

This visualization helps in understanding how well the model performs on individual test images by clearly showing the predicted label and the true label, along with an indication of whether the prediction was correct or not.

Finally, the last part of the code extends the functionality to plot the prediction probabilities for each class alongside the images and their predicted labels. The function plot_value_array is defined to create a bar chart representing the prediction probabilities for each class. The function takes three arguments: the index i, the array of predictions predictions_array, and the true label true_label. The predictions and true label are extracted for the given index, and a bar chart is plotted with the prediction probabilities. The x-axis ticks are set to range from 0 to 9, corresponding to the classes, and the y-axis is limited to the range [0, 1]. The bar representing the predicted class is colored red, while the bar representing the true class is colored blue, providing a visual indication of the model’s performance.

Then, a block of code sets up the plotting of the first 15 test images along with their predicted labels and the true labels. The number of rows and columns for the subplot grid is defined, with 5 rows and 3 columns, resulting in a total of 15 images. A figure is created with a specified size to accommodate the subplots. For each image, two subplots are created side by side: one for the image with its predicted and true labels, and another for the prediction probabilities bar chart. The function plot_image is called to plot the image, and plot_value_array is called to plot the prediction probabilities.

The plt.tight_layout() function is used to ensure that the subplots are neatly arranged without overlapping, and plt.show() displays the combined plots. This comprehensive visualization helps in evaluating the model’s performance by providing a detailed view of the predicted probabilities and how they compare to the true labels for each test image.

Results:

Executing the code in Google Colab took approximately 8 minutes. This time includes the full training process, model evaluation, and the generation of plots. Please note that actual runtime may vary depending on factors such as the available hardware resources and network conditions.

As a result, we achieved an impressive test accuracy of 99.2%. This high level of accuracy indicates that our model is performing exceptionally well in recognizing and classifying the test dataset. Such a result demonstrates the effectiveness of our neural network architecture and the preprocessing steps applied to the data. Achieving this accuracy highlights the potential of our approach in real-world applications, where precise and reliable predictions are crucial.

After 10 epochs, both the training and test losses were impressively low, remaining below 0.05. This indicates that our model has learned the training data very well and is also generalizing effectively to new, unseen data. Low loss values in both training and testing phases suggest that the model is not overfitting and is capable of making accurate predictions. This level of performance demonstrates the robustness of our training process and the suitability of our chosen model architecture for this particular task.

The predicted handwritten numbers are displayed above. With this level of accuracy, there were no mistakes in the predictions. Our CNN model has been thoroughly trained, tested, and validated, demonstrating excellent performance and reliability.

---

Now, armed with these guidelines, go ahead and create your own CNN model! Remember, learning is a journey, so don’t be discouraged by setbacks. Keep experimenting, exploring, and refining your skills. Happy coding!

---

6. Final Thoughts

In summary, Convolutional Neural Networks (CNNs) have emerged as a transformative force in the realm of computer vision and beyond, demonstrating state-of-the-art performance across a diverse range of applications. These include image classification, where CNNs excel at categorizing images into predefined classes, and object detection, where they identify and localize multiple objects within an image. This capability is critical for advancements in fields like autonomous driving and surveillance. Image segmentation, another vital application, involves partitioning an image into segments that correspond to different objects or regions, which is particularly important in medical imaging and scene understanding. Facial recognition systems also rely heavily on CNNs to identify and verify individuals based on facial features, a technology that has been widely adopted in security systems and social media platforms. Additionally, CNNs are employed in gesture recognition, interpreting hand movements and gestures in interactive technologies, virtual reality systems, and video games.

Moreover, in the field of medical image analysis, CNNs assist in diagnosing diseases from medical scans, significantly enhancing the capabilities of radiology and other medical domains. Their remarkable ability to process visually structured data has made them an invaluable tool in these and many other applications.

The versatility of CNNs extends well beyond traditional image recognition tasks. As a multifaceted tool, CNNs are capable of handling any data that can be visually represented, provided the data is encoded in a matrix of pixels or a tensor of voxels. For instance, in sound and voice recognition, audio signals can be transformed into spectrograms. These spectrograms are then processed as images by CNNs to identify and interpret audio patterns, demonstrating their adaptability to different types of data. In financial data analysis, CNNs are used to analyze stock market trends by converting temporal series data into visual formats, such as heatmaps or time-frequency representations. This enables CNNs to detect patterns and make predictions, showcasing their potential in diverse domains beyond the visual spectrum. CNNs continue to revolutionize various fields by leveraging their ability to process and analyze visually structured data. Their application in areas such as medical imaging, sound recognition, and financial data analysis underscores their versatility and power. As a result, CNNs have become an indispensable tool in modern artificial intelligence, driving innovation and enhancing the capabilities of numerous technologies.

Through our comprehensive exploration, we hope to have provided a deeper understanding of CNNs, from their theoretical foundations to their practical applications. By harnessing the transformative potential of CNNs, readers can leverage this groundbreaking technology in their own endeavors, unlocking new possibilities and driving forward the frontiers of artificial intelligence.

#AI #ArtificialIntelligence #CNN #CodingWithChatGPT #ComputerVision #GoogleColab #Keras #LearningConvolutionalNeuralNetworks #Pandas #PatternRecognition #Python #Sandbox #TensorFlow

References:

100+ Machine Learning Projects with Source Code [2024]

Python Machine Learning Projects

Lee, H., Grosse, R., Ranganath, R. and Ng, A.Y., 2009, June. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th annual international conference on machine learning (pp. 609-616).

Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D. and Riedmiller, M., 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602.

Wang, A., Kurutach, T., Liu, K., Abbeel, P. and Tamar, A., 2019. Learning robotic manipulation through visual planning and acting. arXiv preprint arXiv:1905.04411.

Bishop, C.M., 2006. Pattern recognition and machine learning. Springer google schola, 2, pp.1122-1128.

Hearty, J., 2016. Advanced machine learning with Python. Packt Publishing Ltd.

Russell, S., & Norvig, P. (2016). Artificial Intelligence: A Modern Approach. Pearson.

Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction. MIT Press.

Müller, A. C., & Guido, S. (2016). Introduction to Machine Learning with Python: A Guide for Data Scientists. O’Reilly Media.

---

Copyright 2026 AI-Talks.org