Model-free Approaches in Learning the Multivariate Linear Regressive Models

Abstract

Learning from data means to have a learning method that is an algorithm implemented in software that estimates an unknown dependency between a system's inputs and outputs from the available data, namely from known samples. Once such a dependency has been accurately estimated, it can be used for prediction of future system outputs for known input values. The paper provides a series of results concerning the learning from data a linear regressive model in a multivariate framework. The parameter estimates of the regressive model are determined using the maximum likelihood principle and the adaptive learning algorithms are derived using the gradient ascend technique. In the second section of the paper the parameters of the linear regressive model are determined by minimizing the arithmetic mean of square errors and an adaptive learning scheme of gradient descent type is also considered. We consider a probabilistic approach in the third section for modeling the effects of both the latent variables and noise. The cumulative effects of latent variables and noise are modeled in terms of multivariate Gaussian distributions. The predicted output is expressed as the sum of a linear combination of the entries of the input and the random vector that represents the effects of the unobservable factors and noise. The parameters of the regressive model are estimated by maximizing the likelihood function for given finite length sequence of observations, and an adaptive learning algorithm of gradient ascent type is proposed in the final part of the section. A series of concluding remarks and suggestions for further work are formulated in the final section of the paper.