https://ai-talks.org/blog-english-potugues/

Maurício Pinheiro

1. Introduction

1.1. Neurons:

Neurons are the basic building blocks of the nervous system and are responsible for transmitting information throughout the body. They were first discovered and studied by scientists such as Santiago Ramón y Cajal and Camillo Golgi in the late 19th century. The primary function of neurons is to receive, process, and transmit electrochemical signals, which allows them to communicate with other neurons and facilitate various bodily functions. Neurons consist of a cell body, dendrites, and an axon, which all work together to pass signals between cells. The process of communication between neurons is known as synaptic transmission and involves the release of neurotransmitters that bind to receptors on the dendrites of other neurons. In artificial neural networks, neurons are modeled as mathematical functions that take inputs, apply weights and biases, and output a result. These artificial neurons are then connected to form layers, which can be trained using various algorithms such as the perceptron algorithm. With this understanding of neurons and their function, we can appreciate how the perceptron algorithm can emulate the decision-making process of biological neurons in a simplified manner, allowing for the binary classification of data with a simple linear boundary.

*Multipolar Neuron. By BruceBlaus Creative Commons Attribution 3.0 September 30, 2013. Source: Wikimedia Commons.*

1.2. Linear Perceptrons – Artificial Neurons

Linear perceptrons are one of the simplest models of artificial neurons, and they form the basis for more complex neural network architectures. At their core, linear perceptrons are binary classifiers that can learn to separate input data into two classes. They are based on the concept of a threshold function, which separates input data into two classes based on a threshold value.

A linear perceptron receives input data, applies weights to each input x_i, and sums the weighted inputs w_ij to produce an output Σ (net_j). If the output is above a certain threshold θ_j, the perceptron classifies the input into one class o_j, and if it is below the threshold, it classifies the input into another class. The weights are learned during a training phase where the perceptron is presented with labeled examples of input data and adjusts its weights to minimize the classification error. The result is a binary classification.

*Diagram of an artificial neuron model (Linear Perpceptron). By Chrislb, Creative Commons Attribution-Share Alike 3.0. Published on 14. Jul. 2005. Source: Wikimedia Commons.*

The perceptron algorithm was first introduced by Frank Rosenblatt in 1957 as a way of creating a simple model of how neurons in the brain might work. However, it was not until the 1980s that researchers showed that linear perceptrons could be used to solve many practical classification problems. Today, linear perceptrons are widely used in many areas of machine learning and artificial intelligence, including computer vision, natural language processing, and speech recognition.

One example of an application of linear perceptrons is in sorting spam emails. Spam filters often use machine learning algorithms to classify emails as either spam or not spam. Linear perceptrons can be trained on a set of labeled email messages, where the input features correspond to characteristics of the email, such as the sender’s email address, the subject line, and the body of the message. The output of the perceptron is either “spam” or “not spam,” depending on the threshold function. By training the perceptron on a large dataset of labeled email messages, it can learn to accurately classify new messages as either spam or not spam. This is just one example of how linear perceptrons can be used to solve practical classification problems.

As for a problem that this algorithm may be used for, binary classification problems are common applications of linear perceptrons. For example, the algorithm could be used to classify email messages as either spam or not spam, based on the contents of the email. The features used to represent the emails could be word frequencies, presence or absence of certain words, etc. Another example could be image recognition, where the algorithm is trained to distinguish between images of cats and dogs based on pixel values of the images.

2. The Perceptron example

In the following example, I have developed a code for a linear perceptron with the help of ChatGPT and compiled it in a Python IDE. This example served multiple purposes:

First, it showcases how ChatGPT can generate AI code in Python, even if the user is not familiar with the syntax or coding. This tool is amazing as it can create code in any programming language and translate it into a language the user knows, such as Pascal or Basic. ChatGPT is an excellent resource for learning programming and new languages, as it includes “computer” programming languages.
Second, this example serves as a proof of concept of the capabilities of AI-based Large Language models, such as ChatGPT, to create functional AI code, like the perceptron presented here. As the field of AI and machine learning continues to grow, it is possible that some programming tasks may be automated using these models, leading to potential changes in the job market for programmers and developers.
Third the perceptron is a fundamental concept in artificial neural networks and can be applied to solve classification problems. By learning about neurons and perceptrons, we can better understand how neural networks work and how they can be applied in various fields.
Fourth, this was a didatic experiment for me after reading about perceptrons.

2.1. The code

The given Python code is a basic implementation of a perceptron algorithm for binary classification. At first, the required libraries, namely NumPy and Matplotlib, are imported. The Perceptron class is then defined, which contains three methods – init(), predict(), and train().

The init() method initializes the weights for the perceptron with an array of zeros and sets the learning rate for the weights’ update. The predict() method takes an input vector and calculates the dot product with the weight vector and bias term, which is then passed through the step function to predict the class label of the input. The step function returns 1 if the activation is greater than or equal to 0 and -1 otherwise.

The train() method trains the perceptron by iterating over the input and label pairs and updating the weight vector to minimize the errors. It uses the predicted class label and the actual label to calculate the error and adjust the weights. The number of misclassifications in each epoch is stored in a list, which is plotted against the number of epochs. The plot_errors() method plots the training errors as a function of epoch.

The plot_data() method plots the input data points with color-coded labels and the decision boundary determined by the trained perceptron. The training_inputs and labels are generated by NumPy’s random function, and the perceptron is trained using 20 epochs. Finally, the plot_data() method is called to visualize the data points and the decision boundary.

import numpy as np
import matplotlib.pyplot as plt

class Perceptron:
    def __init__(self, input_size, learning_rate=0.1):
        # Initialize the perceptron with an array of weights (input_size + 1), initialized to 0
        self.weights = np.zeros(input_size + 1)
        # Set the learning rate for updating the weights
        self.learning_rate = learning_rate
    
    def predict(self, input_vector):
        # Calculate the dot product of the input vector and the weights, and add the bias term (weights[0])
        activation = np.dot(input_vector, self.weights[1:]) + self.weights[0]
        # Apply the step function to return the predicted class label
        return 1 if activation >= 0 else -1
    
    def train(self, training_inputs, labels, num_epochs):
        # Initialize an empty list to store the number of misclassifications in each epoch
        errors = []
        for epoch in range(num_epochs):
            error = 0
            # Loop through each training input and label
            for input_vector, label in zip(training_inputs, labels):
                # Predict the class label based on the current weights
                prediction = self.predict(input_vector)
                # Calculate the error as the difference between the predicted and actual label
                error += int(label != prediction)
                # Update the weights based on the error and the learning rate
                self.weights[1:] += self.learning_rate * (label - prediction) * input_vector
                self.weights[0] += self.learning_rate * (label - prediction)
            # Append the number of misclassifications for this epoch to the errors list
            errors.append(error)
        # Plot the training errors as a function of epoch
        self.plot_errors(errors, num_epochs)
    
    def plot_errors(self, errors, num_epochs):
        # Plot the training errors as a function of epoch
        plt.plot(range(1, num_epochs+1), errors)
        plt.xlabel('Epoch')
        plt.ylabel('Number of errors')
        plt.title('Training errors over epochs')
        plt.show()
    
    def plot_data(self, training_inputs, labels):
        # Plot the data points with color-coded labels
        plt.figure(figsize=(8,8))
        plt.scatter(training_inputs[:,0], training_inputs[:,1], c=labels, cmap='bwr')
        # Define the plot limits based on the range of the data points
        x_min, x_max = training_inputs[:,0].min() - 1, training_inputs[:,0].max() + 1
        y_min, y_max = training_inputs[:,1].min() - 1, training_inputs[:,1].max() + 1
        # Create a mesh grid for the decision boundary plot
        xx, yy = np.meshgrid(np.arange(x_min, x_max, 0.1), np.arange(y_min, y_max, 0.1))
        Z = np.array([self.predict(np.array([x, y])) for x, y in np.c_[xx.ravel(), yy.ravel()]])
        Z = Z.reshape(xx.shape)
        # Plot the decision boundary as a contour line
        plt.contour(xx, yy, Z, levels=[0], colors='k')
        plt.title('Classification of Data Points')
        plt.xlabel('Feature 1')
        plt.ylabel('Feature 2')
        plt.show()

# Generate some random data points with binary labels
np.random.seed(0)
training_inputs = np.random.randn(100, 2)
labels = np.array([1 if np.dot(x, [1, 2]) + 0.5 > 0 else -1 for x in training_inputs])

# Train a perceptron to classify the data points
perceptron = Perceptron(input_size=2)
perceptron.train(training_inputs, labels, num_epochs=20)

# Plot the data points with the decision boundary determined by the trained perceptron
perceptron.plot_data(training_inputs, labels)

I encourage you to copy this code into the Python IDE and run it. Feel free to modify it and test new possibilities, such as creating a neural network with layers of perceptrons.

2.2. The results

2.2.1. The evolution of the perceptron training

The perceptron algorithm is a simple machine learning algorithm used for binary classification problems. It takes in input data and labels and learns to classify new data points by updating the weights of the perceptron based on the errors made during training. The effectiveness of the algorithm depends on the number of epochs it is trained for. The “epoch” figure generated by the code above shows the training error of the perceptron algorithm over the number of epochs. The figure consists of a line plot with the number of errors on the y-axis and the number of epochs on the x-axis. As the perceptron algorithm is trained over the input data, the number of errors made by the algorithm decreases, and the line plot shows this trend. The plot shows that the number of errors initially starts high and gradually decreases as the number of epochs increases. The figure helps to visualize the training process of the perceptron algorithm and can be used to determine the optimal number of epochs required to achieve a low error rate. The perceptron algorithm can be stopped when the error rate is sufficiently low or when the number of epochs reaches a predetermined maximum.

The y-axis represents the number of errors made by the perceptron algorithm during training, while the x-axis represents the number of epochs. The blue line in the figure represents the number of errors made by the perceptron algorithm over the epochs.

The figure generated by the code above shows the classification of data points using a perceptron algorithm. The figure consists of a scatter plot of 100 two-dimensional data points, with one feature on the x-axis and another feature on the y-axis. The data points are colored red or blue, depending on their class label, which is either +1 or -1. The perceptron algorithm is used to draw a decision boundary that separates the red and blue data points. The decision boundary is shown as a black line, which separates the data points into two regions. All the data points on one side of the line are classified as +1, and all the data points on the other side of the line are classified as -1.

The perceptron algorithm trains the perceptron model on the input data and minimizes the classification error by updating the weights of the model. The training process is shown as a plot of the number of errors made by the perceptron algorithm over the epochs. The plot shows how the number of errors decreases over time as the perceptron algorithm becomes better at classifying the data points. The figure provides a clear visualization of the performance of the perceptron algorithm and can be used to evaluate the accuracy of the model. By analyzing the figure, one can determine if the model is performing well or if further adjustments to the model are necessary. Overall, the figure helps to understand the capabilities and limitations of the perceptron algorithm for classification tasks.

The figure above shows a scatter plot of 100 two-dimensional data points classified using a perceptron algorithm. The red and blue data points represent two different classes, +1 and -1, respectively (that for example, could be important E-mails +1 and Spam -1). The black line in the plot shows the decision boundary generated by the perceptron algorithm that separates the data points into two regions. The training process of the perceptron algorithm is also depicted in the plot, showing the number of errors made by the algorithm over the epochs. As the algorithm is trained, the number of errors decreases, indicating improved classification accuracy.

3. Limitations

While the perceptron algorithm is a powerful and simple method for binary classification problems, it does have some limitations. One major limitation is that it can only separate data points that are linearly separable, which means that the decision boundary must be a straight line or a hyperplane in higher dimensions. If the data is not linearly separable, then the perceptron algorithm will not converge and cannot classify the data accurately.

To overcome this limitation, researchers have developed more advanced neural network models, such as multi-layer perceptrons (MLP) and convolutional neural networks (CNN), which can handle non-linear data and more complex classification tasks. These models have multiple layers of perceptrons and use activation functions that introduce non-linearity into the decision boundary.

Another limitation of the perceptron algorithm is that it can be sensitive to outliers and noisy data, which can cause the algorithm to overfit or underfit the data. To address this issue, regularization techniques, such as L1 and L2 regularization, can be used to control the complexity of the model and prevent overfitting. Additionally, pre-processing techniques, such as feature scaling and data normalization, can be used to reduce the impact of outliers and improve the performance of the algorithm.

4. Conclusions

The perceptron algorithm is a simple yet powerful machine learning algorithm that can be used for binary classification problems. It takes input data and labels and learns to classify new data points by updating the weights of the perceptron based on the errors made during training. The “epoch” figure generated by the code above shows the training error of the perceptron algorithm over the number of epochs, allowing us to visualize the training process and determine the optimal number of epochs required to achieve a low error rate. However, linear perceptrons have their limitations, such as their inability to handle non-linearly separable data. To address this limitation, non-linear activation functions can be used, or more complex models such as multi-layer perceptrons or deep neural networks can be employed. Nonetheless, linear perceptrons remain a valuable and widely used tool in the field of machine learning, and their simplicity and interpretability make them an attractive option for certain applications. With further research and development, we can continue to improve the effectiveness and versatility of the perceptron algorithm and its variants.

In conclusion, this example of developing a linear perceptron with the help of ChatGPT serves as an illustration of the incredible potential of AI-based models in the field of programming and machine learning. By enabling users to generate functional code without prior knowledge of syntax or programming languages, ChatGPT can democratize access to programming and empower more people to engage in this field. Furthermore, this example highlights the importance of understanding fundamental concepts such as perceptrons in the field of artificial neural networks. By gaining a deeper understanding of these concepts, we can better appreciate the potential of neural networks to solve complex problems in various fields, from computer vision to natural language processing. As AI and machine learning continue to evolve and impact our daily lives, it is crucial that we continue to learn and stay informed about the latest developments in these fields.