lambda-ml.neural-network
Multilayer perceptron neural network learning using backpropagation.
Example usage:
(def data [[0 0 [0]] [0 1 [1]] [1 0 [1]] [1 1 [0]]])
(def fit
(let [alpha 0.5
lambda 0.001
model (-> (make-neural-network alpha lambda)
(add-neural-network-layer 2 sigmoid) ;; input layer
(add-neural-network-layer 3 sigmoid) ;; hidden layer
(add-neural-network-layer 1 sigmoid))] ;; output layer
(-> (iterate #(neural-network-fit % data) model)
(nth 5000))))
(neural-network-predict fit (map butlast data))
;;=> [[0.04262340225834812] [0.9582632706756758] [0.9581124103456861] [0.04103544440312673]]
add-neural-network-layer
(add-neural-network-layer model n f)Adds a layer to a neural network model with n nodes and an activation function f.
back-propagate
(back-propagate y theta fns' activations output-error)Returns the errors of each node in a neural network after propagating the the errors at the output nodes, computed against a single target value y, backwards through the network.
compute-gradients
(compute-gradients x activations errors)Returns the gradients for each weight given activation values and errors on a input values of a single example x.
feed-forward
(feed-forward x theta fns)Returns the activation values for nodes in a neural network after forward propagating the values of a single input example x through the network.
feed-forward-batch
(feed-forward-batch x theta fns)Returns the activation values for nodes in a neural network after forward propagating a collection of input examples x through the network.
gradient-descent
(gradient-descent model x y)Performs gradient descent on input and target values of all examples x and y, and returns the updated weights.
gradient-descent-step
(gradient-descent-step x y theta fns alpha lambda cost output-error)Performs a single gradient step on the input and target values of a single example x and label y, and returns the updated weights.
make-neural-network
(make-neural-network alpha lambda)(make-neural-network alpha lambda cost)Returns a neural network model where alpha is the learning rate.
neural-network-fit
(neural-network-fit model data)(neural-network-fit model x y)Trains a neural network model for the given training data. For new models, parameters are initialized as random values from a normal distribution.
neural-network-predict
(neural-network-predict model x)Predicts the values of example data using a neural network model.
numeric-gradients
(numeric-gradients x y theta fns cost)Returns the numeric approximations of the gradients for each weight given the input values of a single example x and label y. Used for debugging by checking against the computed gradients during backpropagation.
print-neural-network
(print-neural-network model)Prints information about a given neural network.