The role of loss functions in regression problems
Options
BORIS DOI
Abstract
In regression analysis, the goal is to capture the influence of one or more explanatory
variables X1, . . . ,Xm on a response variable Y in terms of a regression function
g : Rm -> R. An estimate ĝ of g is then found or evaluated in terms of its ability to predict
a prespecified statistical functional T of the conditional distribution L(Y |X1, . . . ,Xm).
This is done with the help of a loss function that penalizes estimates that perform poorly
in predicting T(L(Y |X1, . . . ,Xm)). More precisely, it is done by using loss functions that
are consistent for T.
Clearly, the outcome of the evaluation or estimation strongly depends on the functional
T. However, when we focus on a specific functional T a vast collection of suitable loss
functions may be available and the result can still be sensitive to the choice of loss
function.
There are several viable solution strategies to approach this issue. We can, for instance,
impose additional properties on the loss function or the resulting estimate so that only
one of the possible loss functions remains reasonable. In this doctoral thesis we adopt
another approach. The underlying idea is that we would naturally prefer an estimate ĝ
that is optimal with respect to several consistent loss functions for T, as then the choice
of loss function seems to impact the outcome less severely.
In Chapter 1, we consider the nonparametric isotonic regression problem. We show that
this regression problem is special in that for identifiable functionals T, solutions which
are simultaneously optimal with respect to an entire class of consistent losses exist and
can be characterized. There are, however, several functionals of interest that are not
identifiable. The expected shortfall is just one prominent example. However, some of
those functionals can be obtained as a function of a vector-valued elicitable functional.
In the second Chapter, we investigate when simultaneous optimality with respect to a
class of consistent losses holds for these functionals and introduce the solution to the
isotonic regression problem for a specific loss in the case where simultaneous optimality
is not fulfilled.
In parametric regression, on the other hand, different consistent loss functions often yield
different parameter estimates under misspecification. This motivates to consider the set
of these parameters as a way to measure misspecification. We introduce this approach
in Chapter 3 and show how the set of these model parameters can be calculated on the
population and on the sample level for an isotonic regression function g.
variables X1, . . . ,Xm on a response variable Y in terms of a regression function
g : Rm -> R. An estimate ĝ of g is then found or evaluated in terms of its ability to predict
a prespecified statistical functional T of the conditional distribution L(Y |X1, . . . ,Xm).
This is done with the help of a loss function that penalizes estimates that perform poorly
in predicting T(L(Y |X1, . . . ,Xm)). More precisely, it is done by using loss functions that
are consistent for T.
Clearly, the outcome of the evaluation or estimation strongly depends on the functional
T. However, when we focus on a specific functional T a vast collection of suitable loss
functions may be available and the result can still be sensitive to the choice of loss
function.
There are several viable solution strategies to approach this issue. We can, for instance,
impose additional properties on the loss function or the resulting estimate so that only
one of the possible loss functions remains reasonable. In this doctoral thesis we adopt
another approach. The underlying idea is that we would naturally prefer an estimate ĝ
that is optimal with respect to several consistent loss functions for T, as then the choice
of loss function seems to impact the outcome less severely.
In Chapter 1, we consider the nonparametric isotonic regression problem. We show that
this regression problem is special in that for identifiable functionals T, solutions which
are simultaneously optimal with respect to an entire class of consistent losses exist and
can be characterized. There are, however, several functionals of interest that are not
identifiable. The expected shortfall is just one prominent example. However, some of
those functionals can be obtained as a function of a vector-valued elicitable functional.
In the second Chapter, we investigate when simultaneous optimality with respect to a
class of consistent losses holds for these functionals and introduce the solution to the
isotonic regression problem for a specific loss in the case where simultaneous optimality
is not fulfilled.
In parametric regression, on the other hand, different consistent loss functions often yield
different parameter estimates under misspecification. This motivates to consider the set
of these parameters as a way to measure misspecification. We introduce this approach
in Chapter 3 and show how the set of these model parameters can be calculated on the
population and on the sample level for an isotonic regression function g.
Date of Publication
2021
Year of graduation
2021
Theses Type
dissertation
Subject(s)
Language(s)
en
Author(s)
Mühlemann, Anja |
Faculty/Graduate School
Access(Rights)
open.access
Primary OA Publication
true