What's so special about Artificial Neural Networks?

What's so special about Artificial Neural Networks? Aren't they just some advanced regression functions?

Not only! The special thing about Artificial Neural Networks is the fact that they are "universial approximators". This means that any arbitrary functional dependency can be rebuild using an Artifical Neural Network, as long as there are enough neurons used in the hidden layers.

Roughly spoken: "If you can't do it with Neural Networks, you also will have to fight very hard with any other method."

In this sense Artificial Neural Networks are Data Mining tools, which can be used to uncover functional dependencies and relationionships of parameters within given data sets.

When doing mathematical modeling it's mainly about finding a function f, which describes the relation between the system inputs x and the system outputs y in a proper way: y = f(x). Hereby following subtasks have to be solved:

1) Definition of the input and output variables x and y in a way that the system behaviour is described sufficiently with them,

2) specification of examples (x₁,y₁), (x₂,y₂), ... , (x_n,y_n), for which the function should be valid,

3) finding out a function f, that y_i = f(x_i) for all i from 2)

Within the conventional analytical procedures one first thinks about a draft function f for the variables x and y. With this function as working hypothesis then the examples of 2) are proofed to be valid. That way the hypothesis of the function can only be verified or falsified. In the case of falsification one normaly has to go back and think about a new hypothesis, in the extreme case go back to start. For complex problems the danger of runing into extensive and expensive trial and error scenarios then is very present, because one never knows which of the 3 issues above (the coding of the problem 1), the functional requirements 2) or the mathematical function 3) is the reason for failing.

An alternative approch would be taking an artificial neural network first as draft for the function in point 3). Artificial Neural Networks are universial approximators, which can reproduce any possible functional relationship. Hereby the maximum possible functional performance can be achieved. This eliminates the the issue 3) in the first run, allowing to concentrate on the solution of the issues 1) and 2). If no neural network can be found to resolve 1) and 2) satisfying, it is very likely that the choice of parameters x and/or y and is not done well. Or the data 2) are not building a functional relationship with the parametrisation x and y of 1), meaning that the requirements for the mathematical model are unfeasible and unrealistic. 

For example the data set in 2) contains a pair of data (x_i,y_i), (x_j,y_j), where the input x_i is equal or very similar to x_j but the outputs y_i and y_j are different. We are talking of a conflict in the functional requirements then.

The resolution of such a conflict could either be the extension of the input vector x with new coefficients/parameters, which allow distinguishing between x_i and x_j then. Or one resigns example i or j from the functional requirements. The search for an analytical function f is only continued for 3) after the issues 1) and 2) are resolved accordingly. If the problem is so complex that it can only be solved with the help of a Artificial Neural Network, the given neural network can immediatly be taken for implementation. Otherwise one can continue with the search for an elegant analytical function for f. Hereby the neural networks can as well be helpful for the estimation of the problem complexity and the chance of really finding a analytical solution. For complex problems this top-down approach is much more efficient and leads to a solution much quicker.

The often heard statement of "experts" that neural networks are not working in their field cannot be valid! In this case the failure is mainly due to a weak problem formulation and wrong coding of the problem (issue 1) or due to unrealistic requirements (issue 2). Very seldomly it is due to the neural networks (the function in 3).

A further aspect is the availability of Artificial Neural Networks. Also other universial approximators are existing in the field of Machine Learning for which the above arguments are valid as well. But currently neural networks are the most elaborated and best available. A lot of good, professional and free software products are offering Artificial Neural Networks. Not much needs to be invented or developed any more and one can concentrate on the primary engineering task.