d given a set of data D = (xi , yi )n i =1 , where x ∈ R , y ∈ R
the prediction of a new sample x by D, i.e., y (x|D) or P (x|D)
Mingmin Chi (Fudan Univ.)
Intro to ML
Lecture for ML
10 / 35
Learning Machine
Function Approximation
If exists a mapping between inputs x and outputs y , the prediction can be obtained by function approximation, i.e., y := f (x, w) What’s the form of f ? How to estimate w?
Intro to ML
Lecture for ML
4 / 35
Learning Machine
Definition
Machine Learning: is the field of study that gives computers the ability to learn without being explicitly programmed. [Samuel, 1959] is a science of the artificial. The field’s main objects of study are artifacts, specifically algorithms that improve their performance with experience. [Langley, 1996] is the study of computer algorithms that improve automatically through experience. [Mitchell, 1997] is programming computers to optimize a performance criterion using example data or past experience. [Alpaydin, 2004] is a scientific discipline that is concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data, such as from sensor data or databases. [Wikipedia, 2010]
Introduction
Mingmin Chi
Fudan University, Shanghai, China
Mingmin Chi (Fudan Univ.)
Intro to ML
Lecture for ML
1 / 35
Outline
1
Learning Machine Setting of Learning Problem Decision Theory Other related issue
Mingmin Chi (Fudan Univ.)
Intro to ML
Lecture for ML
3 / 35
Learning Machine
Learning ... What?
rules functions behaviors abilities knowledge ...
Mingmin Chi (Fudan Univ.)
- One standard formulation of the supervised learning task is the classification problem: the learner is required to learn (to approximate) the behavior of a function which maps a vector xi ∈ RD into one of several classes by looking at several input-output examples of the function
Learning Types
Imagine a machine which experiences a series of sensory inputs: xi , i = 1, · · · , n Supervised learning: in which the algorithm generates a function that maps inputs xi to desired outputs yi .
Mingmin Chi (Fudan Univ.)
Intro to ML
Lecture for ML
12 / 35
Learning Machine
Typical Learning Diagram
Mingmin Chi (Fudan Univ.)
Intro to ML
Lecture for ML
13 / 35
Mingmin Chi (Fudan Univ.)
Intro to ML
Lecture for ML
6 / 35
Learning Machine
Examples?
Mingmin Chi (Fudan Univ.)
Intro to ML
Lecture for ML
7 / 35
Learning Machine
Mingmin Chi (Fudan Univ.) Intro to ML Lecture for ML 14 / 35
Learning Machine
Inference Types
Inductive Learning (specific-to-general): Learning is a problem of function estimation on the basis of empirical data. [Vapnik pp. 291]
Learning Machine
Learning Types (cont’d)
Imagine a machine which experiences a series of sensory inputs: xi , i = 1, · · · , n Unsupervised learning: is to build a model of xi that can be used for reasoning, decision making, predicting things, communicating etc. Labeled examples are not available. Semi-supervised learning: which combines both labeled and unlabeled examples to generate an appropriate function or classifier Reinforcement learning: in which the algorithm learns a policy of how to act given an observation of the world. Every action ai has some impact in the environment, and the environment provides feedback (rewards or punishments) that guides the learning algorithm. Its goal is to learn to act in a way that maximizes rewards in the long term
Mingmin Chi (Fudan Univ.)
Intro to ML
Lecture for ML
9 / 35
Learning Machine
Supervised Learning
Components for learning in common a set of variables –> inputs x, which are measured or preset one or more outputs (responses) y the goal is to use the inputs to predict the values of the outputs x−> y Supervised learning
Mingmin Chi (Fudan Univ.)
Intro to ML
Lecture for ML
15 / 35
Learning Machine
Inference Types
Inductive Learning (specific-to-general): Learning is a problem of function estimation on the basis of empirical data. [Vapnik pp. 291] Transductive Learning (specific-to-specific): To estimate the values of the function for a given finite number of samples of interest. [Vapnik pp. 292]
2
3
4
Mingmin Chi (Fudan Univ.)
Intro to ML