Representation vs Learning:
- Representation: whether or not a function can be simulated by the model; i.e. is the model capable of representing a given function?
- Learning: whether or not their exists an algorithm with which the weights can be adjusted to represent a particular function
Types of learning
- supervised learning - the learning algorithm is provided with pre-labeled training examples to learn from.
- unsupervised learning - the learning algorithm is provided with unlabeled examples. Generally, unsupervised learning is used to uncover some structure of or pattern in the data.
- semi-supervised learning - the learning algorithm is provided with a mixture of labeled and unlabeled data.
- active learning - similar to semi-supervised learning, but the algorithm can "ask" for extra labeled data based on what it needs to improve on.
- reinforcement learning - actions are taken and rewarded or penalized in some way and the goal is maximizing lifetime/long-term reward (or minimizing lifetime/long-term penalty).
- Neural Computing: Theory and Practice (1989). Philip D. Wasserman