Overview

Neural networks are a branch of machine learning algorithms inspired by the structure and function of the human brain. They recognize patterns and relationships by processing data through a set of interconnected nodes; called neurons.

Neural networks consist of neuron layers that receive inputs from the previous layer and pass outputs on to the subsequent layer. The neural network feeds input data into the first (input) layer, and the result of the final (output) layer constitutes the algorithm’s final prediction or classification.

During training, the neural network adapts the strength of connections between neurons, called weights, to optimize performance on a given task with respect to the loss. This is called backpropagation. Backpropagation involves computing the error between the network’s output and the desired output, then using the error to update the weight matrices of connections between neurons. Traditional neural networks use some gradient descent algorithm to optimize the loss. This process is repeated for many epochs until the neural network reaches the optimal weights, according to the gradient of the loss function.

Neural network applications span multiple fields, such as image and speech recognition, natural language processing, and predictive modeling. They are particularly well-equipped for handling large amounts of complex data sources. In these domains, neural networks often outperform other traditional machine learning algorithms.

Data Prep

Supervised machine learning algorithms, such as neural networks, require labeled data because they learn from already classified or labeled examples. The goal is to discover the mapping between the input data and the corresponding output labels. In supervised learning, the algorithm tries to find patterns in the labeled data to make predictions or classifications on new, unlabeled data.

The training set is the portion of the data used to train or build the model, while the testing set is the set that evaluates the model’s performance. The testing set is usually a subset of the data the model has not seen during training. The ‘train_test_split’ function within the Sci-Kit Learn library partitioned the data into training and testing sets. The test set contained 25% of the data (42 countries), and the remaining 75% constituted the training set (127 countries). The training and testing sets are necessarily disjoint to prevent the model from memorizing the training data instead of learning the underlying patterns. If the testing set overlaps with the training set, the model may perform well on the test set but fail to generalize to new, unseen data. This failure to generalize to new data is called overfitting. Therefore, splitting the data into training and testing sets is vital to ensuring that the model is trained on one set and tested on another. The train-test split allows the model to learn from the training set while providing an unbiased evaluation of its performance on the testing set.

Code

Results

Model hyper-parameters

• Epochs: 3000
• Activation Function: Leaky ReLU
• Loss Function: Categorical Cross entropy
• Optimizer: Gradient Descent
• Learning rate (eta): 0.01

Conclusions

These results show that even a simple artificial neural network, with only one hidden layer, can accurately predict aid allocation levels of developing countries. The ANN with the specified hyper-parameters above achieved a prediction accuracy of 88.1% on the testing set. These results are another example of the application of machine learning algorithms in foreign aid research. The high performance of this simple model allows for the possibility of even better performance in more complex neural networks.