ARCHIVED PROJECT · C++ & AI
Revisiting My Early C++ Neural Network Project
Use the archived project description as a starting point for understanding how a small network learns from examples.
The Programs page on my old website listed a C++ project called Perceptron Network. The archived description says it used nine inputs and eight outputs, started with small random connection strengths, and adjusted weights with a delta learning rule. After training, the user could test the network with different inputs. It was described as a Windows program.
That short description is enough to identify the idea behind the project, but not enough to recover every implementation decision. This article revisits the concepts and supplies a new, much smaller teaching example. The code below is not the recovered original program. It uses two inputs and one linear output so that the weight updates remain easy to inspect.
What the archive tells us—and what it does not
The original summary describes training with input/output pairs and continuing until the results were good enough. It does not document the exact activation function, the training examples, the stopping threshold or whether the network contained hidden layers. Those details affect how the model behaves. I am leaving them unspecified rather than filling in a plausible design and presenting it as historical fact.
The name Perceptron Network also deserves care. A classic perceptron classifier and a linear unit trained with a delta rule are related introductory models, but they do not use identical learning procedures. The archived title alone does not resolve that distinction. Looking at the original source would let us explain the exact variant; for now, the delta-rule description is the clearest documented starting point.
A weight controls how much an input contributes
In a simple linear neuron, each input is multiplied by a weight and the results are added to a bias. The bias lets the output shift even when the inputs are zero. For two inputs, the prediction is w1*x1 + w2*x2 + b. A positive weight increases the contribution of that input, while a negative weight reverses its influence.
The teaching task uses a known target relationship: 2*x1 - x2 + 0.5. We intentionally know the answer because that gives us something concrete to check. The model begins with zero weights and bias, then learns from four examples. Starting at zero is fine for this single linear unit; it should not be generalized to all multilayer-network initialization.
The delta rule moves the prediction toward the target
For each example, compute the target minus the current prediction. Multiply that error by an input and a learning rate, then add the result to the corresponding weight. Update the bias using the same error without an input multiplier. Here, the procedure is an online gradient update for half the squared error of a linear output.
An example with a zero input does not directly change the corresponding weight, because the product is zero. The bias can still change. This is a useful observation when you debug the loop: a weight is not adjusted merely because the prediction is wrong. Its update depends on the input that contributed to the prediction.
The learning rate determines the size of each step. A large rate can produce unstable behavior, while a very small rate may need many passes through the examples. Our modest input values and fixed rate make the demonstration straightforward. A model with different feature scales needs its own analysis instead of copying these settings uncritically.
Run a new compact C++ example
Save this as linear-neuron.cpp. Compile it with a C++17 compiler using g++ -std=c++17 linear-neuron.cpp -o linear-neuron, then run the resulting executable. The program uses standard-library facilities and no Windows-specific interface. You can download the full teaching example.
#include <array>
#include <cmath>
#include <iostream>
struct Example {
std::array<double, 2> input;
double target;
};
int main() {
const std::array<Example, 4> training{{
{{{0.0, 0.0}}, 0.5},
{{{1.0, 0.0}}, 2.5},
{{{0.0, 1.0}}, -0.5},
{{{1.0, 1.0}}, 1.5}
}};
std::array<double, 2> weights{{0.0, 0.0}};
double bias = 0.0;
const double rate = 0.05;
auto predict = [&](const std::array<double, 2>& input) {
return weights[0] * input[0] + weights[1] * input[1] + bias;
};
for (int epoch = 0; epoch < 2000; ++epoch) {
for (const auto& example : training) {
const double error = example.target - predict(example.input);
for (std::size_t j = 0; j < weights.size(); ++j) {
weights[j] += rate * error * example.input[j];
}
bias += rate * error;
}
}
double mse = 0.0;
for (const auto& example : training) {
const double error = example.target - predict(example.input);
mse += error * error;
}
mse /= training.size();
const std::array<double, 2> unseen{{0.25, 0.75}};
const double expected = 0.25;
std::cout << "Weights: " << weights[0] << ", " << weights[1]
<< " Bias: " << bias << " MSE: " << mse << '\n';
std::cout << "Unseen input prediction: " << predict(unseen) << '\n';
return mse < 1e-8 && std::abs(predict(unseen) - expected) < 1e-4 ? 0 : 1;
}
The expected learned parameters are approximately two, minus one and one-half. The unseen input is one-quarter and three-quarters, which has a target of one-quarter. The program returns a failure code if the training error or the unseen prediction fails its stated tolerance. This checks a known synthetic relationship; it does not measure performance on a real classification problem.
Training error is only one part of the story
Mean squared error summarizes how far the predictions are from the targets in this tiny training set. It is useful here because the target relationship is linear and noise-free. If the data were noisy, contradictory or generated by a nonlinear relationship, repeatedly training the same simple unit would not necessarily remove the remaining error.
The unseen example provides a separate numerical check, but it is still drawn from the same deliberately constructed rule. A practical project needs a more representative evaluation collection. The questions become: does the model behave sensibly on inputs it was not trained on, how does it handle unusual values, and what mistakes matter in the intended use?
A network does not become nonlinear by having more outputs
Multiple output units can each predict a different target, but adding more linear outputs does not by itself produce nonlinear decision boundaries. A stack of linear operations without nonlinear activation still amounts to a linear transformation. Architecture and activation choices matter as much as the number of nodes.
A classic introductory limitation is XOR: one linear threshold boundary cannot separate its positive and negative cases in the original two-dimensional input space. Additional nonlinear features or a model with appropriate nonlinear hidden units changes the situation. That example explains a capability boundary; it is not evidence that the archived nine-input, eight-output program was intended to solve XOR.
Why preserve a small early AI project?
A modest neural-network experiment makes the mechanics of learning visible. You can print the weights, step through one update and check the effect of a changed training target. That is a useful foundation before using larger libraries where much of the numerical work happens behind an API. The small scale is an educational advantage rather than something to disguise.
The next historical step is to recover the original files and document their actual structure. Until then, the archived summary and the new teaching code remain clearly separated. Read machine learning versus AI for context. Source references are the 2004 Programs archive and Cornell’s perceptron lecture, which discusses the classic classifier rather than claiming to describe my original code.