Editor’s note: One of the central technologies of artificial intelligence is neural networks. In this interview, Tam Nguyen, a professor of computer science at the University of Dayton, explains how neural networks, programs in which a series of algorithms try to simulate the human brain work.

1. Artificial Neural Network Examples
2. Neural Network C++ Code Example
3. Neural Network Algorithm Example

Neural networks approach the problem in a different way. The idea is to take a large number of handwritten digits, known as training examples, and then develop a system which can learn from those training examples. In other words, the neural network uses the examples to automatically infer rules for recognizing handwritten digits. I am currently trying to create a neural network to predict poker hands, I am quite new to machine learning and neural networks and might need some help!I found some tutorials on how to create a neural network.

What are some examples of neural networks that are familiar to most people?

1. Opponent Modeling and Exploitation in Poker Using Evolved Recurrent Neural Networks (2018) Xun Li As a classic example of imperfect information games, poker, in particular, Heads-Up No-Limit Texas Holdem (HUNL), has been studied extensively in recent years.
2. Artificial neural networks (ANNs) are software implementations of the neuronal structure of our brains. We don’t need to talk about the complex biology of our brain structures, but suffice to say, the brain.

There are many applications of neural networks. One common example is your smartphone camera’s ability to recognize faces.

Driverless cars are equipped with multiple cameras which try to recognize other vehicles, traffic signs and pedestrians by using neural networks, and turn or adjust their speed accordingly.

Neural networks are also behind the text suggestions you see while writing texts or emails, and even in the translations tools available online.

Does the network need to have prior knowledge of something to be able to classify or recognize it?

Yes, that’s why there is a need to use big data in training neural networks. They work because they are trained on vast amounts of data to then recognize, classify and predict things.

In the driverless cars example, it would need to look at millions of images and video of all the things on the street and be told what each of those things is. When you click on the images of crosswalks to prove that you’re not a robot while browsing the internet, it can also be used to help train a neural network. Only after seeing millions of crosswalks, from all different angles and lighting conditions, would a self-driving car be able to recognize them when it’s driving around in real life.

More complicated neural networks are actually able to teach themselves. In the video linked below, the network is given the task of going from point A to point B, and you can see it trying all sorts of things to try to get the model to the end of the course, until it finds one that does the best job.

Some neural networks can work together to create something new. In this example, the networks create virtual faces that don’t belong to real people when you refresh the screen. One network makes an attempt at creating a face, and the other tries to judge whether it is real or fake. They go back and forth until the second one cannot tell that the face created by the first is fake.

Humans take advantage of big data too. A person perceives around 30 frames or images per second, which means 1,800 images per minute, and over 600 million images per year. That is why we should give neural networks a similar opportunity to have the big data for training.

How does a basic neural network work?

A neural network is a network of artificial neurons programmed in software. It tries to simulate the human brain, so it has many layers of “neurons” just like the neurons in our brain. The first layer of neurons will receive inputs like images, video, sound, text, etc. This input data goes through all the layers, as the output of one layer is fed into the next layer.

Let’s take an example of a neural network that is trained to recognize dogs and cats. The first layer of neurons will break up this image into areas of light and dark. This data will be fed into the next layer to recognize edges. The next layer would then try to recognize the shapes formed by the combination of edges. The data would go through several layers in a similar fashion to finally recognize whether the image you showed it is a dog or a cat according to the data it’s been trained on.

These networks can be incredibly complex and consist of millions of parameters to classify and recognize the input it receives.

Why are we seeing so many applications of neural networks now?

Actually neural networks were invented a long time ago, in 1943, when Warren McCulloch and Walter Pitts created a computational model for neural networks based on algorithms. Then the idea went through a long hibernation because the immense computational resources needed to build neural networks did not exist yet.

Recently, the idea has come back in a big way, thanks to advanced computational resources like graphical processing units (GPUs). They are chips that have been used for processing graphics in video games, but it turns out that they are excellent for crunching the data required to run neural networks too. That is why we now see the proliferation of neural networks.

[Understand new developments in science, health and technology, each week.Subscribe to The Conversation’s science newsletter.]

Recurrent Neural Network(RNN) are a type of Neural Network where the output from previous step are fed as input to the current step. In traditional neural networks, all the inputs and outputs are independent of each other, but in cases like when it is required to predict the next word of a sentence, the previous words are required and hence there is a need to remember the previous words. Thus RNN came into existence, which solved this issue with the help of a Hidden Layer. The main and most important feature of RNN is Hidden state, which remembers some information about a sequence.

RNN have a “memory” which remembers all information about what has been calculated. It uses the same parameters for each input as it performs the same task on all the inputs or hidden layers to produce the output. This reduces the complexity of parameters, unlike other neural networks.

How RNN works

The working of a RNN can be understood with the help of below example:

Example:

Suppose there is a deeper network with one input layer, three hidden layers and one output layer. Then like other neural networks, each hidden layer will have its own set of weights and biases, let’s say, for hidden layer 1 the weights and biases are (w1, b1), (w2, b2) for second hidden layer and (w3, b3) for third hidden layer. This means that each of these layers are independent of each other, i.e. they do not memorize the previous outputs.

Now the RNN will do the following:

• RNN converts the independent activations into dependent activations by providing the same weights and biases to all the layers, thus reducing the complexity of increasing parameters and memorizing each previous outputs by giving each output as input to the next hidden layer.
• Hence these three layers can be joined together such that the weights and bias of all the hidden layers is the same, into a single recurrent layer.
• Formula for calculating current state:

  
  where:

• Formula for applying Activation function(tanh):

  
  where:

• Formula for calculating output:

Training through RNN

1. A single time step of the input is provided to the network.
2. Then calculate its current state using set of current input and the previous state.
3. The current ht becomes ht-1 for the next time step.
4. One can go as many time steps according to the problem and join the information from all the previous states.
5. Once all the time steps are completed the final current state is used to calculate the output.
6. The output is then compared to the actual output i.e the target output and the error is generated.
7. The error is then back-propagated to the network to update the weights and hence the network (RNN) is trained.

Advantages of Recurrent Neural Network

Artificial Neural Network Examples

1. An RNN remembers each and every information through time. It is useful in time series prediction only because of the feature to remember previous inputs as well. This is called Long Short Term Memory.
2. Recurrent neural network are even used with convolutional layers to extend the effective pixel neighborhood.

Neural Network C++ Code Example

Disadvantages of Recurrent Neural Network

Neural Network Algorithm Example

1. Gradient vanishing and exploding problems.
2. Training an RNN is a very difficult task.
3. It cannot process very long sequences if using tanh or relu as an activation function.

Recommended Posts:

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the 'Improve Article' button below.