Objective: Implement a single neuron, then a full multi-layer forward pass, using nothing but raw NumPy arrays and matrix multiplication. All in runnable, in-browser Python.
Task 1: A Single Neuron
Implement the neuron equation directly: z = w·x + b, then apply a sigmoid activation.
import numpy as np
def sigmoid(z):
return 1 / (1 + np.exp(-z))
def neuron(x, w, b):
z = np.dot(w, x) + b
return sigmoid(z)
x = np.array([2.0, -1.0, 0.5])
w = np.array([0.4, 0.6, -0.2])
b = 0.1
output = neuron(x, w, b)
print("Pre-activation z:", np.dot(w, x) + b)
print("Neuron output (after sigmoid):", round(float(output), 4))
Task 2: A Layer of Neurons
A layer is several neurons evaluated against the same input. Stack their weight vectors into a matrix
W of shape (n_out, n_in) and compute the whole layer in one matrix-vector product.
import numpy as np
def sigmoid(z):
return 1 / (1 + np.exp(-z))
def relu(z):
return np.maximum(0, z)
def layer_forward(x, W, b, activation):
z = W @ x + b
return activation(z), z
x = np.array([2.0, -1.0, 0.5])
np.random.seed(42)
W = np.random.randn(4, 3) * 0.5 # 4 neurons, each taking a 3-dim input
b = np.zeros(4)
a, z = layer_forward(x, W, b, relu)
print("W shape:", W.shape)
print("Pre-activations z:", np.round(z, 3))
print("Activations a (after ReLU):", np.round(a, 3))
Confirm for yourself: W has 4 rows, one per neuron in the layer, and 3 columns, matching the
3-dimensional input x. The output a has length 4, one activation per neuron.
Task 3: Chain Two Layers Into a Small Network
Feed the first layer's activations as the second layer's input. This is the entire forward-propagation
algorithm: repeat layer_forward once per layer, threading the output of each into the next.
import numpy as np
def sigmoid(z):
return 1 / (1 + np.exp(-z))
def relu(z):
return np.maximum(0, z)
def layer_forward(x, W, b, activation):
z = W @ x + b
return activation(z), z
np.random.seed(42)
# Architecture: 3 inputs -> hidden layer of 4 -> output layer of 1 (binary classification)
W1, b1 = np.random.randn(4, 3) * 0.5, np.zeros(4)
W2, b2 = np.random.randn(1, 4) * 0.5, np.zeros(1)
def network_forward(x):
a1, z1 = layer_forward(x, W1, b1, relu)
y_hat, z2 = layer_forward(a1, W2, b2, sigmoid)
return y_hat, {"a1": a1, "z1": z1, "z2": z2}
x = np.array([2.0, -1.0, 0.5])
y_hat, cache = network_forward(x)
print("Hidden layer activations:", np.round(cache["a1"], 3))
print("Predicted probability: ", round(float(y_hat[0]), 4))
The dictionary returned alongside y_hat, conventionally called a cache, holds
every intermediate value computed along the way. This is exactly the information backpropagation needs in
the next lab; storing it now means you won't have to recompute the forward pass during the backward pass.
Task 4: Run a Batch of Examples at Once
Real training never processes one example at a time; it processes a batch. With NumPy, this means
x becomes a matrix of shape (n_in, batch_size) instead of a single vector, and the
exact same matrix-multiply code handles the whole batch with no changes beyond the bias broadcasting correctly.
import numpy as np
def relu(z):
return np.maximum(0, z)
def sigmoid(z):
return 1 / (1 + np.exp(-z))
np.random.seed(42)
W1, b1 = np.random.randn(4, 3) * 0.5, np.zeros((4, 1))
W2, b2 = np.random.randn(1, 4) * 0.5, np.zeros((1, 1))
# A batch of 5 examples, each with 3 features, stored as columns
X_batch = np.random.randn(3, 5)
Z1 = W1 @ X_batch + b1 # shape (4, 5): broadcasts b1 across all 5 columns
A1 = relu(Z1)
Z2 = W2 @ A1 + b2 # shape (1, 5)
Y_hat = sigmoid(Z2)
print("X_batch shape:", X_batch.shape)
print("Hidden activations shape:", A1.shape)
print("Predictions for all 5 examples:", np.round(Y_hat.flatten(), 4))
This is precisely why deep learning leans so heavily on matrix multiplication and GPUs: the same code that handles one example handles a batch of thousands, and that batched matrix multiplication is exactly what GPU hardware is built to accelerate, a theme picked up again in the practical training phase later in this course.