* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Package `FCNN4R`
Survey
Document related concepts
Transcript
Package ‘FCNN4R’ March 9, 2016 Type Package Title Fast Compressed Neural Networks for R Version 0.6.2 Date 2016-03-08 Author Grzegorz Klima <[email protected]> Maintainer Grzegorz Klima <[email protected]> Description Provides an interface to kernel routines from the FCNN C++ library. FCNN is based on a completely new Artificial Neural Network representation that offers unmatched efficiency, modularity, and extensibility. FCNN4R provides standard teaching (backpropagation, Rprop, simulated annealing, stochastic gradient) and pruning algorithms (minimum magnitude, Optimal Brain Surgeon), but it is first and foremost an efficient computational engine. Users can easily implement their algorithms by taking advantage of fast gradient computing routines, as well as network reconstruction functionality (removing weights and redundant neurons, reordering inputs, merging networks). Networks can be exported to C functions in order to integrate them into virtually any software solution. Depends R (>= 3.0), stats, graphics, methods, Rcpp License GPL (>= 2) NeedsCompilation yes Repository CRAN Date/Publication 2016-03-09 00:57:57 R topics documented: FCNN4R-package . . . . . . . . . . . is.mlp_net . . . . . . . . . . . . . . . mlp_eval . . . . . . . . . . . . . . . . mlp_export_C . . . . . . . . . . . . . mlp_net . . . . . . . . . . . . . . . . mlp_net-absolute-weight-indices . . . mlp_net-accessing-individual-weights . . . . . . . . . . . . . . 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 4 4 5 6 7 2 FCNN4R-package mlp_net-class . . . . . . . . . . . . . mlp_net-combining-two-networks . . mlp_net-display . . . . . . . . . . . . mlp_net-export-import . . . . . . . . mlp_net-general-information . . . . . mlp_net-manipulating-network-inputs mlp_net-MSE-gradients . . . . . . . . mlp_net-names . . . . . . . . . . . . mlp_net-weights-access . . . . . . . . mlp_plot . . . . . . . . . . . . . . . . mlp_prune_mag . . . . . . . . . . . . mlp_prune_obs . . . . . . . . . . . . mlp_rm_neurons . . . . . . . . . . . mlp_rnd_weights . . . . . . . . . . . mlp_set_activation . . . . . . . . . . mlp_teach_bp . . . . . . . . . . . . . mlp_teach_grprop . . . . . . . . . . . mlp_teach_rprop . . . . . . . . . . . mlp_teach_sa . . . . . . . . . . . . . mlp_teach_sgd . . . . . . . . . . . . read-write-fcnndataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Index FCNN4R-package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 9 10 10 11 12 13 14 15 16 16 17 18 19 19 20 21 23 24 25 26 28 Fast Compressed Neural Networks for R Description Provides an interface to kernel routines from the FCNN C++ library. FCNN is based on a completely new Artificial Neural Network representation that offers unmatched efficiency, modularity, and extensibility. FCNN4R provides standard teaching (backpropagation, Rprop, simulated annealing, stochastic gradient) and pruning algorithms (minimum magnitude, Optimal Brain Surgeon), but it is first and foremost an efficient computational engine. Users can easily implement their algorithms by taking advantage of fast gradient computing routines, as well as network reconstruction functionality (removing weights and redundant neurons, reordering inputs, merging networks). Networks can be exported to C functions in order to integrate them into virtually any software solution. Author(s) Grzegorz Klima <[email protected]> References G. Klima. A new approach towards implementing artificial neural networks. Technical Report, http://fcnn.sourceforge.net/, 2013. is.mlp_net 3 Examples # set up the XOR problem inputs and outputs inp <- c(0, 0, 1, 1, 0, 1, 0, 1) dim(inp) <- c(4, 2) outp <- c(0, 1, 1, 0) dim(outp) <- c(4, 1) # create a 2-6-1 network net <- mlp_net(c(2, 6, 1)) # set activation function in all layers net <- mlp_set_activation(net, layer = "a", "sigmoid") # randomise weights net <- mlp_rnd_weights(net) # tolerance level tol <- 0.5e-4 # teach using Rprop, assign trained network and plot learning history netmse <- mlp_teach_rprop(net, inp, outp, tol_level = tol, max_epochs = 500, report_freq = 10) net <- netmse$net plot(netmse$mse, type = 'l') # plot network with weights mlp_plot(net, TRUE) # if the algorithm had converged, prune using Optimal Brain Surgeon and plot if (mlp_mse(net, inp, outp) <= tol) { net <- mlp_prune_obs(net, inp, outp, tol_level = tol, max_reteach_epochs = 500, report = TRUE)[[1]] mlp_plot(net, TRUE) } # check network output round(mlp_eval(net, inp), digits = 3) is.mlp_net Is it an object of mlp_net class? Description This function checks whether argument is an object of mlp_net class. Usage is.mlp_net(x) Arguments x Value Logical value. an object to be checked 4 mlp_export_C mlp_eval Evaluation Description Evaluate network output. Usage mlp_eval(net, input) Arguments net an object of mlp_net class input numeric matrix, each row corresponds to one input vector, the number of columns must be equal to the number of neurons in the network input layer Value Numeric matrix with rows representing network outputs corresponding to input rows. mlp_export_C Export multilayer perceptron network to a C function Description This function exports multilayer perceptron network to a C function with optional affine input and output transformations: Ax+b for inputs and Cx+d for outputs. Usage mlp_export_C(fname, net, with_bp = FALSE, A = NULL, b = NULL, C = NULL, d = NULL) Arguments fname character string with the filename net an object of mlp_net class with_bp logical, should backpropagation code for online learning be exported? A numeric matrix (optional), input linear transformation b numeric vector (optional), input translation C numeric matrix (optional), output linear transformation d numeric vector (optional), output translation mlp_net 5 Value Logical value, TRUE if export was successful, FALSE otherwise. Examples # create a 2-3-1 network net <- mlp_net(c(2, 3, 1)) # randomise weights net <- mlp_rnd_weights(net) # show the network show(net) # export network to a C function mlp_export_C("test.c", net) # show the output file file.show("test.c") mlp_net Create objects of mlp_net class Description Function used for creating multilayer perceptron networks. Usage mlp_net(layers, name = NULL) Arguments layers vector providing numbers of neurons in each layer name character string, network name (optional) Value Returns an object of mlp_net class. See Also mlp_net for details. 6 mlp_net-absolute-weight-indices Examples # create a 2-3-1 network net <- mlp_net(c(2, 3, 1)) # randomise weights net <- mlp_rnd_weights(net) # show basic information about the network show(net) mlp_net-absolute-weight-indices Retrieving absolute weight index Description In some situations absolute weight index (i.e. index within all weights including inactive ones) needs to be computed based on information about connected neurons’ indices or weight index within actives ones. The latter functionality is especially useful in implementation of pruning algorithms. Usage mlp_get_w_idx(net, layer, nidx, nplidx) mlp_get_w_abs_idx(net, idx) Arguments net an object of mlp_net class layer integer value (vector), layer index nidx integer value (vector), neuron index nplidx integer value (vector), index of the neuron in the previous layer determining connection from neuron nidx in layer, 0 denotes bias of neuron nidx in layer idx integer value (vector), weight index (indices) within active ones Value Absolute weight index. mlp_net-accessing-individual-weights 7 mlp_net-accessing-individual-weights Setting and retrieving status (on/off) and value of individual weight(s) Description The following functions can be used to access individual weight(s), i.e. set or retrieve status(es) (on/off) and value(s). Usage mlp_set_w_st(net, on, idx = NULL, layer = NULL, nidx = NULL, nplidx = NULL) mlp_set_w(net, val, idx = NULL, layer = NULL, nidx = NULL, nplidx = NULL) mlp_get_w_st(net, idx = NULL, layer = NULL, nidx = NULL, nplidx = NULL) mlp_get_w(net, idx = NULL, layer = NULL, nidx = NULL, nplidx = NULL) Arguments net an object of mlp_net class on logical value (vector), should the weight be set on or off? idx integer value (vector), weight absolute index layer integer value (vector), layer index nidx integer value (vector), neuron index nplidx integer value (vector), index of the neuron in the previous layer determining connection from neuron nidx in layer, 0 denotes bias of neuron nidx in layer val numeric value (vector), connection (or bias) value to be set Value mlp_set_w_st returns network (an object of mlp_net class) with state(s) (on/off) of selected weight(s) set. mlp_set_w returns network (an object of mlp_net class) with value(s) of selected weight(s) set. mlp_get_w_st returns logical value (vector), TRUE if connection/bias is active, FALSE otherwise. mlp_get_w returns numeric value (vector), selected weight value(s). 8 mlp_net-class mlp_net-class An S4 class representing Multilayer Perception Network. Description The mlp_net class represents the Multilayer Perception Network employing the so-called compressed representation, which was inspired by the Compressed Column Storage familiar from sparse matrix algebra. Although the representation and algorithms working with it are somewhat complicated, the user is provided with a simple and intuitive interface that completely hides the internal workings of the package, which in its large part is written in C++. Slots m_name character string, network name m_layers integer vector, stores the numbers of neurons in layers m_n_pointers integer vector, stores the so-called ’pointers’ to neurons m_n_prev integer vector, stores the number of connected neurons in the previous layer m_n_next integer vector, stores the number of connected neurons in the next layer m_w_pointers integer vector, stores the so-called ’pointers’ to weights m_w_values numeric vector, values of connection weights and biases m_w_flags logical vector, states (active/inactive) of weights and biases m_w_on integer value, the number of active weights m_af integer vector, activation functions’ indices m_af_p numeric vector, activation functions’ slope parameters References G. Klima. A new approach towards implementing artificial neural networks. Technical Report, http://fcnn.sourceforge.net/, 2013. See Also mlp_net for creating objects of this class. mlp_net-combining-two-networks 9 mlp_net-combining-two-networks Combining two networks into one Description These functions construct new network by merging two networks (they must have the same number of layers) or by connecting one network outputs to another network inputs (the numbers of output and input neurons must agree). These functions may be used in constructing deep learning networks or constructing networks with some special topologies. Usage mlp_merge(net1, net2, same_inputs = FALSE) mlp_stack(net1, net2) Arguments net1 an object of mlp_net class net2 an object of mlp_net class same_inputs logical, if TRUE both merged networks are assumed to take the same inputs (they share the input layer), default is FALSE Value Both functions return an object of mlp_net class. Examples # create two 2-2-2 networks with random weights and plot them net1 <- mlp_net(c(2, 2, 2)) net1 <- mlp_rnd_weights(net1) mlp_plot(net1, TRUE) net2 <- mlp_net(c(2, 2, 2)) net2 <- mlp_rnd_weights(net2) mlp_plot(net2, TRUE) # create a 4-3-2 network with random weights and plot it net3 <- mlp_net(c(4, 3, 2)) net3 <- mlp_rnd_weights(net3) mlp_plot(net3, TRUE) # construct new network using net1, net2, and net3 and plot it net4 <- mlp_stack(mlp_merge(net1, net2), net3) mlp_plot(net4, TRUE) 10 mlp_net-export-import mlp_net-display Displaying networks (objects of mlp_net class) Description These methods can be used to display objects of mlp_net class. show and print provide short information about network structure and activation functions, summary gives detailed information about all network connections. Usage ## S4 method for signature 'mlp_net' show(object) ## S4 method for signature 'mlp_net' print(x) ## S4 method for signature 'mlp_net' summary(object) Arguments object an object of mlp_net class x an object of mlp_net class mlp_net-export-import Export and import multilayer perceptron network to/from a text file in FCNN format Description These functions can be used to export and import multilayer perceptron network to/from a text file in FCNN format. Usage mlp_export_fcnn(fname, net) mlp_import_fcnn(fname) Arguments fname character string with the filename net an object of mlp_net class mlp_net-general-information 11 Details Files are organised as follows: • the first comment (beginning with #) is treated as network information (name) string, • all other comments are ignored, • network structure is represented by five block of numbers: – – – – the first line determines numbers of neurons in consecutive layers, the second block of 0’s and 1’s determines which weights are turned off/on, the third block contains active weights’ values, the last block determines hidden and output layers’ activation functions and their slope parameters - each line contains 2 numbers: the function index and its slope parameter. Value mlp_export_fcnn returns logical value, TRUE if export was successful, FALSE otherwise. mlp_import_fcnn returns an object of mlp_net class or NULL, if import failed. See Also mlp_net for network representation details. Examples # create a 2-3-1 network net <- mlp_net(c(2, 3, 1)) # randomise weights net <- mlp_rnd_weights(net) # show the network show(net) # export network mlp_export_fcnn("test.net", net) # show the output file file.show("test.net") # import network net2 <- mlp_import_fcnn("test.net") # show the imported network show(net2) mlp_net-general-information General information about network Description The following functions return basic information about the network. 12 mlp_net-manipulating-network-inputs Usage mlp_get_layers(net) mlp_get_no_active_w(net) mlp_get_no_w(net) Arguments net an object of mlp_net class Value mlp_get_layers returns an integer vector with numbers of neurons in consecutive layers. mlp_get_no_active_w returns the number of active weights (connections and biases). mlp_get_no_w returns the total number (including inactive) of weights (connections and biases). See Also mlp_net-class for details on internal network representation. mlp_net-manipulating-network-inputs Manipulating network inputs Description These functions construct new network by removing redundant (i.e. not connected to the next layer) inputs or reordering / expanding network inputs. Usage mlp_rm_input_neurons(net, report = FALSE) mlp_expand_reorder_inputs(net, newnoinputs, inputsmap) Arguments net an object of mlp_net class report logical value, if TRUE, information about removed neurons will be printed on the console (FALSE by default) newnoinputs integer value, determines the number of inputs in the new network inputsmap integer vector, determines the mapping of old inputs into new ones - the ith value of this vector will be the new index of ith input mlp_net-MSE-gradients 13 Value mlp_rm_input_neurons returns a two-element list. The first element (net) is the network (an object of mlp_net class) with all redundant input neurons removed, the second (ind) - the indices of input neurons that were not removed. mlp_expand_reorder_inputs returns an object of mlp_net class. Examples # construct a 2-4-3 network, plot result nn <- mlp_net(c(2, 4, 3)) nn <- mlp_rnd_weights(nn) mlp_plot(nn, TRUE) # expand inputs, the new no. of inputs will be 5, with the first input # becoming the 3rd and the second retaining its position, plot result nn <- mlp_expand_reorder_inputs(nn, 5, c(3, 2)) mlp_plot(nn, TRUE) # remove redundant neurons (i.e. 1, 4, 5) and plot result nn <- mlp_rm_input_neurons(nn, TRUE)$net mlp_plot(nn, TRUE) mlp_net-MSE-gradients Computing mean squared error, its gradient, and output derivatives Description The functions use fast FCNN kernel routines and are intended for implementing teaching and pruning algorithms. Usage mlp_mse(net, input, output) mlp_grad(net, input, output) mlp_gradi(net, input, output, i) mlp_gradij(net, input, i) mlp_jacob(net, input, i) Arguments net an object of mlp_net class input numeric matrix, each row corresponds to one input vector, the number of columns must be equal to the number of neurons in the network input layer 14 mlp_net-names output numeric matrix with rows corresponding to expected outputs, the number of columns must be equal to the number of neurons in the network output layer, the number of rows must be equal to the number of input rows i data row index Details mlp_mse returns the mean squared error (MSE). MSE is understood as half of the squared error averaged over all outputs and data records. mlp_grad computes the gradient of MSE w.r.t. network weights. This function is useful when implementing batch teaching algorithms. mlp_gradi computes the gradient of MSE w.r.t. network weights at the ith data record. This is normalised by the number of outputs only, the average over all rows (all i) returns the same as grad(input, output). This function is useful for implementing on-line teaching algorithms. mlp_gradij computes gradients of network outputs, i.e the derivatives of outputs w.r.t. active weights, at given data row. The derivatives of outputs are placed in subsequent columns of the returned matrix. Scaled by the output errors and averaged they give the same as gradi(input, output, i). This function is useful in implementing teaching algorithms using second order corrections and Optimal Brain Surgeon pruning algorithm. mlp_jacob computes the Jacobian of network outputs, i.e the derivatives of outputs w.r.t. inputs, at given data row. The derivatives of outputs are placed in subsequent columns of the returned matrix. Value mlp_mse returns mean squared error (numeric value). mlp_grad returns two-element lists with the first field (grad) containing numeric vector with gradient and the second (mse) - the mean squared error. mlp_gradi returns numeric vector with gradient. mlp_gradij returns numeric matrix with gradients of outputs in consecutive columns. mlp_jacob returns numeric matrix with derivatives of outputs in consecutive columns. mlp_net-names Get and set network names Description The following functions can be used for retrieving and setting network names. Usage mlp_get_name(net) mlp_set_name(net, name) mlp_net-weights-access 15 Arguments net an object of mlp_net class name character string with network name Value mlp_get_name returns character string with network name. mlp_set_name returns network (an object of mlp_net class) with name set to new value. mlp_net-weights-access Set and retrieve (active) weights’ values Description One of FCNN’s design objectives (and main advantages) is the complete separation of teaching (and pruning) algorithms from internal network structure workings. This goal is achieved through fast access to (active) weights vector facilitated by FCNN’s ‘compressed’ network representation. The following two functions allow users to efficiently retrieve and set network (active) weights vector. Usage mlp_set_weights(net, weights) mlp_get_weights(net) Arguments net an object of mlp_net class weights numeric vector of new active weights’ values Value mlp_set_weights returns network (an object of mlp_net class) with active weights set to given values. mlp_set_weights returns numeric vector of active weights’ values. 16 mlp_prune_mag mlp_plot Plotting multilayer perceptron network Description This function plots a multilayer perceptron network’s structure. Optionally, weights’ values are displayed on graph. Usage mlp_plot(net, show_weights = FALSE, show_neuron_idx = TRUE) Arguments net an object of mlp_net class show_weights logical, should weights’ values be displayed? (FALSE by default) show_neuron_idx logical, should neurons’ indices be displayed? (TRUE by default) Value This function does not return value. mlp_prune_mag Minimum magnitude pruning Description Minimum magnitude pruning is a brute force, easy-to-implement pruning algorithm in which in each step the weight with the smallest absolute value is turned off. This algorithm requires reteaching network in almost every step and yields suboptimal results. Usage mlp_prune_mag(net, input, output, tol_level, max_reteach_epochs, report, plots = FALSE) Arguments net an object of mlp_net class input numeric matrix, each row corresponds to one input vector, the number of columns must be equal to the number of neurons in the network input layer output numeric matrix with rows corresponding to expected outputs, the number of columns must be equal to the number of neurons in the network output layer, the number of rows must be equal to the number of input rows mlp_prune_obs 17 tol_level numeric value, error (MSE) tolerance level max_reteach_epochs integer value, maximal number of epochs (iterations) allowed when reteaching network report logical value, if TRUE, information about the pruning process will be printed on the console (FALSE by default) plots logical value, if TRUE, the initial network is plotted and then replotted every time neuron is removed and at the end of pruning (FALSE by default) Value Three-element list, the first field (net) contains the pruned network, the second (wcount) - the number of connections removed (inactivated), the third (ncount) - the number of neurons removed. mlp_prune_obs Optimal Brain Surgeon pruning Description The Optimal Brain Surgeon algorithm is a robust (yet computationally demanding) pruning algorithm in which candidate weight to be turned off is determined based on information about the inverse of (approximate) Hessian matrix of the MSE. Usage mlp_prune_obs(net, input, output, tol_level, max_reteach_epochs, report, plots = FALSE, alpha = 1e-05) Arguments net an object of mlp_net class input numeric matrix, each row corresponds to one input vector, the number of columns must be equal to the number of neurons in the network input layer output numeric matrix with rows corresponding to expected outputs, the number of columns must be equal to the number of neurons in the network output layer, the number of rows must be equal to the number of input rows tol_level numeric value, error (MSE) tolerance level max_reteach_epochs integer value, maximal number of epochs (iterations) allowed when reteaching network report logical value, if TRUE, information about the pruning process will be printed on the console (FALSE by default) plots logical value, if TRUE, the initial network is plotted and then replotted every time neuron is removed and at the end of pruning (FALSE by default) alpha numeric value, scaling factor used for initial Hessian approximation 18 mlp_rm_neurons Value Three-element list, the first field (net) contains the pruned network, the second (wcount) - the number of connections removed (inactivated), the third (ncount) - the number of neurons removed. References B. Hassibi, D. G. Stork, and G. J. Wolff. Optimal Brain Surgeon and General Network Pruning. Technical Report CRC-TR-9235, RICOH California Research Centre, 1992. mlp_rm_neurons Remove redundant neurons in a multilayer perceptron network Description This function removes redundant neurons from the network, i.e. hidden layers’ neurons that are not connected to neurons in the previous layer or the next layer. If a neuron is not connected to neurons in the previous layer but is connected to neurons in the next layer (effectively acts as an additional bias), biases of neurons in the next layer are properly adjusted, therefore, the resulting network behaves just like the initial one. Usage mlp_rm_neurons(net, report = FALSE) Arguments net an object of mlp_net class report logical value, if TRUE, information about removed neurons will be printed on the console (FALSE by default) Value Three-element list. The first element (net) is the network (an object of mlp_net class) with all redundant neurons removed, the second (ncount) - the number of neurons removed, the third (wcount) - the number of weights removed. mlp_rnd_weights 19 mlp_rnd_weights This function sets network weights to random values drawn from uniform distribution. Description This function sets network weights to random values drawn from uniform distribution. Usage mlp_rnd_weights(net, a = 0.2) Arguments net a an object of mlp_net class numeric value, values will be drawn from uniform distribution on [-a, a] (by default a = 0.2) Value Network (an object of mlp_net class) with randomised weights. mlp_set_activation Set network activation functions Description This function sets activation function (and its slope parameter) for neurons in the hidden layers and in the output layer. Usage mlp_set_activation(net, layer, activation = c("threshold", "sym_threshold", "linear", "sigmoid", "sym_sigmoid", "tanh", "sigmoid_approx", "sym_sigmoid_approx"), slope = 0) Arguments net layer activation slope an object of mlp_net class integer vector or character value, index (indices) of layer(s) whose activation function will be changed or character: "a" denotes all layers, "h" - hidden layer(s), "o" - the output layer character string, activation function name, admissible options are: "threshold", "sym_threshold", "linear", "sigmoid", "sym_sigmoid" (and "tanh"), "sigmoid_approx", and "sym_sigmoid_approx" numeric value, activation function slope parameter, if 0 the default parameter value is chosen for each activation function 20 mlp_teach_bp Value This function returns network (an object of mlp_net class) with activation function set. mlp_teach_bp Backpropagation (batch) teaching Description Backpropagation (a teaching algorithm) is a simple steepest descent algorithm for MSE minimisation, in which weights are updated according to (scaled) gradient of MSE. Usage mlp_teach_bp(net, input, output, tol_level, max_epochs, learn_rate = 0.7, l2reg = 0, report_freq = 0) Arguments net an object of mlp_net class input numeric matrix, each row corresponds to one input vector, the number of columns must be equal to the number of neurons in the network input layer output numeric matrix with rows corresponding to expected outputs, the number of columns must be equal to the number of neurons in the network output layer, the number of rows must be equal to the number of input rows tol_level numeric value, error (MSE) tolerance level max_epochs integer value, maximal number of epochs (iterations) learn_rate numeric value, learning rate in the backpropagation algorithm (default 0.7) l2reg numeric value, L2 regularization parameter (default 0) report_freq integer value, progress report frequency, if set to 0 no information is printed on the console (this is the default) Value Two-element list, the first field (net) contains the trained network, the second (mse) - the learning history (MSE in consecutive epochs). Note The name ‘backpropagation’ is commonly used in two contexts, which sometimes causes confusion. Firstly, backpropagation can be understood as an efficient algorithm for MSE gradient computation that was first described by Bryson and Ho in the ’60s of 20th century and reinvented in the ’80s. Secondly, the name backpropagation is (more often) used to refer to the steepest descent method that uses gradient of MSE computed efficiently by means of the aforementioned algorithm. This ambiguity is probably caused by the fact that in practically all neural network implementations, the derivatives of MSE and weight updates are computed simultaneously in one backward pass (from output layer to input layer). mlp_teach_grprop 21 References A.E. Bryson and Y.C. Ho. Applied optimal control: optimization, estimation, and control. Blaisdell book in the pure and applied sciences. Blaisdell Pub. Co., 1969. David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning representations by back-propagating errors. Nature, 323(6088):533-536, October 1986. mlp_teach_grprop Rprop teaching - minimising arbitrary objective function Description This implementation (‘generalisation’) of the Rprop algorithm allows users to teach network to minimise arbitrary objective function provided that functions evaluating objective and computing gradient are provided. Usage mlp_teach_grprop(net, obj_func, gradient, epochs, stop = NULL, report_freq = 0, report_action = NULL, u = 1.2, d = 0.5, gmax = 50, gmin = 1e-06) Arguments net an object of mlp_net class obj_func function taking an object of mlp_class class as a single argument returning objective to be minimised gradient function taking an object of mlp_class class as a single argument returning gradient of the objective epochs integer value, number of epochs (iterations) stop function (or NULL), a function taking objective history to date and returning Boolean value (if TRUE is returned, algorithm stops) (the default is not to stop until all iterations are performed) report_freq integer value, progress report frequency, if set to 0 no information is printed on the console (this is the default) report_action function (or NULL), additional action to be taken while printing progress reports, this should be a function taking network as a single argument (default NULL) u numeric value, Rprop algorithm parameter (default 1.2) d numeric value, Rprop algorithm parameter (default 0.5) gmax numeric value, Rprop algorithm parameter (default 50) gmin numeric value, Rprop algorithm parameter (default 1e-6) 22 mlp_teach_grprop Value Two-element list, the first field (net) contains the trained network, the second (obj) - the learning history (value of the objective function in consecutive epochs). References M. Riedmiller. Rprop - Description and Implementation Details: Technical Report. Inst. f. Logik, Komplexitat u. Deduktionssysteme, 1994. Examples ## Not run: # set up XOR problem inp <- c(0, 0, 1, 1, 0, 1, 0, 1) dim(inp) <- c(4, 2) outp <- c(0, 1, 1, 0) dim(outp) <- c(4, 1) # objective obj <- function(net) { return(mlp_mse(net, inp, outp)) } # gradient grad <- function(net) { return(mlp_grad(net, inp, outp)$grad) } # stopping citerion tol <- function(oh) { if (oh[length(oh)] <= 5e-5) { return(TRUE); } return(FALSE) } # create a 2-6-1 network net <- mlp_net(c(2, 6, 1)) # set activation function in all layers net <- mlp_set_activation(net, layer = "a", "sigmoid") # randomise weights net <- mlp_rnd_weights(net) # teach netobj <- mlp_teach_grprop(net, obj, grad, epochs = 500, stop = tol, report_freq = 1) # plot learning history plot(netobj$obj, type = 'l') ## End(Not run) mlp_teach_rprop mlp_teach_rprop 23 Rprop teaching Description Rprop is a fast and robust adaptive step method based on backpropagation. For details, please refer to the original paper given in References section. Usage mlp_teach_rprop(net, input, output, tol_level, max_epochs, l2reg = 0, u = 1.2, d = 0.5, gmax = 50, gmin = 1e-06, report_freq = 0) Arguments net an object of mlp_net class input numeric matrix, each row corresponds to one input vector, the number of columns must be equal to the number of neurons in the network input layer output numeric matrix with rows corresponding to expected outputs, the number of columns must be equal to the number of neurons in the network output layer, the number of rows must be equal to the number of input rows tol_level numeric value, error (MSE) tolerance level max_epochs integer value, maximal number of epochs (iterations) l2reg numeric value, L2 regularization parameter (default 0) u numeric value, Rprop algorithm parameter (default 1.2) d numeric value, Rprop algorithm parameter (default 0.5) gmax numeric value, Rprop algorithm parameter (default 50) gmin numeric value, Rprop algorithm parameter (default 1e-6) report_freq integer value, progress report frequency, if set to 0 no information is printed on the console (this is the default) Value Two-element list, the first field (net) contains the trained network, the second (mse) - the learning history (MSE in consecutive epochs). References M. Riedmiller. Rprop - Description and Implementation Details: Technical Report. Inst. f. Logik, Komplexitat u. Deduktionssysteme, 1994. 24 mlp_teach_sa mlp_teach_sa Teaching networks using Simulated Annealing Description This function can be used to teach an ANN to minimise arbitrary objective function. Usage mlp_teach_sa(net, obj_func, Tinit = 1, epochs = 1000, report_freq = 0, report_action = NULL) Arguments net an object of mlp_net class obj_func function taking an object of mlp_class class as a single argument returning objective to be minimised Tinit numeric value, initial temperature (default is 1) epochs integer value, number of epochs (iterations) (default is 1000) report_freq integer value, progress report frequency, if set to 0 no information is printed on the console (this is the default) report_action function (or NULL), additional action to be taken while printing progress reports, this should be a function taking network as a single argument (default NULL) Value Two-element list, the first field (net) contains the trained network, the second (obj) - the learning history (value of the objective function in consecutive epochs). Examples ## Not run: # set up XOR problem inp <- c(0, 0, 1, 1, 0, 1, 0, 1) dim(inp) <- c(4, 2) outp <- c(0, 1, 1, 0) dim(outp) <- c(4, 1) # objective obj <- function(net) { return(mlp_mse(net, inp, outp)) } # create a 2-6-1 network net <- mlp_net(c(2, 6, 1)) # set activation function in all layers net <- mlp_set_activation(net, layer = "a", "sigmoid") mlp_teach_sgd 25 # teach netobj <- mlp_teach_sa(net, obj, Tinit = 1, epochs = 1000, report_freq = 1) # plot learning history plot(netobj$obj, type = 'l') ## End(Not run) mlp_teach_sgd Stochastic gradient descent with (optional) RMS weights scaling, weight decay, and momentum Description This function implements the stochastic gradient descent method with optional modifications: L2 regularization, root mean square gradient scaling, weight decay, and momentum. Usage mlp_teach_sgd(net, input, output, tol_level, max_epochs, learn_rate, l2reg = 0, minibatchsz = 100, lambda = 0, gamma = 0, momentum = 0, report_freq = 0) Arguments net an object of mlp_net class input numeric matrix, each row corresponds to one input vector number of columns must be equal to the number of neurons in the network input layer output numeric matrix with rows corresponding to expected outputs, number of columns must be equal to the number of neurons in the network output layer, number of rows must be equal to the number of input rows tol_level numeric value, error (MSE) tolerance level max_epochs integer value, maximal number of epochs (iterations) learn_rate numeric value, (initial) learning rate, depending on the problem at hand, learning rates of 0.001 or 0.01 should give satisfactory convergence l2reg numeric value, L2 regularization parameter (default 0) minibatchsz integer value, the size of the mini batch (default 100) lambda numeric value, rmsprop parameter controlling the update of mean squared gradient, reasonable value is 0.1 (default 0) gamma numeric value, weight decay parameter (default 0) momentum numeric value, momentum parameter, reasonable values are between 0.5 and 0.9 (default 0) report_freq integer value, progress report frequency, if set to 0 no information is printed on the console (this is the default) 26 read-write-fcnndataset Value Two-element list, the first field (net) contains the trained network, the second (mse) - the learning history (MSE in consecutive epochs). read-write-fcnndataset Reading and writing datasets in the FCNN format Description These functions can be used to read and write datasets from/to a text file in the FCNN format. Datasets in the similar FANN format (comments are not supported by FANN) can also be read by read.fcnndataset. Usage read.fcnndataset(fname) write.fcnndataset(fname, input, output) Arguments fname character string with the filename input numeric matrix, each row corresponds to one input vector output numeric matrix with rows corresponding to expected outputs, the number of rows must be equal to the number of input rows Details Files are organised as follows: • The first comment (beginning with #) is the dataset information (ignored on read), • three numbers determine: number of records, no. of inputs and no. of outputs, • each data record has two or three lines: – (optional) record information in a comment (beginning with #), – line with input values, – line with output values. Value read.fcnndataset returns a dataframe. write.fcnndataset does not return. read-write-fcnndataset Examples # set up the XOR problem inputs and outputs inp <- c(0, 0, 1, 1, 0, 1, 0, 1) dim(inp) <- c(4, 2) outp <- c(0, 1, 1, 0) dim(outp) <- c(4, 1) # write dataset write.fcnndataset("xor.dat", inp, outp) # show the output file file.show("xor.dat") # read dataset xordf <- read.fcnndataset("xor.dat") # show the imported dataset show(xordf) 27 Index mlp_get_w_abs_idx (mlp_net-absolute-weight-indices), 6 mlp_get_w_idx (mlp_net-absolute-weight-indices), 6 mlp_get_w_st (mlp_net-accessing-individual-weights), 7 mlp_get_weights (mlp_net-weights-access), 15 mlp_grad (mlp_net-MSE-gradients), 13 mlp_gradi (mlp_net-MSE-gradients), 13 mlp_gradij (mlp_net-MSE-gradients), 13 mlp_import_fcnn (mlp_net-export-import), 10 FCNN4R-package, 2 mlp_jacob (mlp_net-MSE-gradients), 13 mlp_merge is.mlp_net, 3 (mlp_net-combining-two-networks), 9 mlp_eval, 4 mlp_mse (mlp_net-MSE-gradients), 13 mlp_expand_reorder_inputs mlp_net, 5, 5, 8, 11 (mlp_net-manipulating-network-inputs), mlp_net-absolute-weight-indices, 6 12 mlp_net-accessing-individual-weights, mlp_export_C, 4 7 mlp_export_fcnn mlp_net-class, 8 (mlp_net-export-import), 10 mlp_net-combining-two-networks, 9 mlp_get_layers mlp_net-display, 10 (mlp_net-general-information), mlp_net-export-import, 10 11 mlp_net-general-information, 11 mlp_get_name (mlp_net-names), 14 mlp_net-manipulating-network-inputs, mlp_get_no_active_w 12 (mlp_net-general-information), mlp_net-method (mlp_net-class), 8 11 mlp_net-MSE-gradients, 13 mlp_get_no_w mlp_net-names, 14 (mlp_net-general-information), mlp_net-weights-access, 15 11 mlp_plot, 16 mlp_get_w mlp_prune_mag, 16 (mlp_net-accessing-individual-weights), 7 mlp_prune_obs, 17 ∗Topic classes is.mlp_net, 3 mlp_net, 5 mlp_net-class, 8 ∗Topic package FCNN4R-package, 2 ∗Topic pruning mlp_prune_mag, 16 mlp_prune_obs, 17 ∗Topic teaching mlp_teach_bp, 20 mlp_teach_grprop, 21 mlp_teach_rprop, 23 mlp_teach_sa, 24 mlp_teach_sgd, 25 28 INDEX mlp_rm_input_neurons (mlp_net-manipulating-network-inputs), 12 mlp_rm_neurons, 18 mlp_rnd_weights, 19 mlp_set_activation, 19 mlp_set_name (mlp_net-names), 14 mlp_set_w (mlp_net-accessing-individual-weights), 7 mlp_set_w_st (mlp_net-accessing-individual-weights), 7 mlp_set_weights (mlp_net-weights-access), 15 mlp_stack (mlp_net-combining-two-networks), 9 mlp_teach_bp, 20 mlp_teach_grprop, 21 mlp_teach_rprop, 23 mlp_teach_sa, 24 mlp_teach_sgd, 25 print,mlp_net-method (mlp_net-display), 10 read-write-fcnndataset, 26 read.fcnndataset (read-write-fcnndataset), 26 show,mlp_net-method (mlp_net-display), 10 summary,mlp_net-method (mlp_net-display), 10 write.fcnndataset (read-write-fcnndataset), 26 29