[Osx-nutters] Forecasting

[David Cake]

At 9:23 PM +0100 20/8/07, Stefano Mori wrote:
>OK, found it.
>
>On 2007-Aug-20, at 06:49, David Cake wrote:
>
>>	But when you hear that the research discipline is primarily a 
>>  combination of management, behavioural sciences, social sciences... 
>>  and other fields, you might begin to think that its not exactly 
>>  what we are looking for here. Management, behavioural sciences, 
>>  social sciences all have relatively limited application to the 
>>  climate, which stubbornly resists being managed, or prediction by 
>>  the rules of psychology or sociology.
>>	Now, human reaction to predictions, constructing appropriate 
>>  plans, etc is an important part of the debate, but not the part of 
>>  the debate we expect the IPCC to deal with.
>
>
>
>The field of forecasting finds that complex models tend to perform 
>poorly and give misleading conclusions.
>
>How is this not of interest to the IPCC??

	I'm sure it is of interest to the IPCC, which is why they 
look at a range of models. But bear in mind that most of the 
forecasting folks research, by their own claims, comes from 
behavioural science, social science, etc. Complex models work JUST 
FINE in the physical sciences a lot of the time. Vague 
generalisations about words like complex are interesting but very far 
from decisive.
	And remember, the IPCC has plenty of simple models that say 
the climate is getting warmer too. If you'd rather trust those simple 
ones more.....

>  >>  > In 2007, the Intergovernmental Panel on Climate Change's Working
>>>>   Group One, a panel of experts established by the World  
>>>>  Meteorological Organization and the United Nations Environment  
>>>>  Programme, issued its Fourth Assessment Report. The Report 
>>>>  included  predictions of dramatic increases in average world 
>>>>  temperatures  over the next 92 years and serious harm resulting 
>>>>  from the  predicted temperature increases. Using forecasting 
>>>>  principles as  our guide we asked: Are these forecasts a good 
>>>>  basis for developing  public policy? Our answer is "no." To 
>>>>  provide forecasts of climate  change that are useful for policy-
>>>>  making, one would need to  forecast (1) global temperature, (2) 
>>>>  the effects of any temperature  changes, (3) the effects of 
>>>>  alternative policies, and (4) whether  the best policy would be 
>>>>  successfully implemented. Proper forecasts  of all four are 
>>>>  necessary for rational policy making.
>>>>
>>
>>	And here we see the issue demonstrated exactly.
>>	They say that the IPCC report isn't a good basis for public 
>>  policy, but their primary reason is because the IPCC report doesn't 
>>  include the effects of temperature changes (and I think there is an 
>>  implicit 'including cost, economic and otherwise' when you are 
>>  talking public policy), and details alternative policies, etc. In a 
>>  report whose brief clearly isn't to report on these things, but 
>>  only on the trends of global temperature.
>
>
>
>Er, that's not the primary reason

	Its the first, and most decisive, reason they quote and 
effectively a summary of the rest, as far as I can tell. When they've 
already defined their terms of the test such that they are guaranteed 
to fail, I'm not inclined to give them the benefit of the doubt that 
their other imprecise criticisms will prove better.

>  and you don't refute the paper by 
>just responding to that warm-up discussion section. Did you not read 
>the rest?

	Yes, but most of it is more or less along the same lines.

>This is key:
>
>>>  We audited the forecasting processes described in Chapter 8 of the
>>>  IPCC's WG1 Report to assess the extent to which they complied with
>>>  forecasting principles. We found enough information to make
>>>  judgments on 89 out of a total of 140 forecasting principles. The
>>>  forecasting procedures that were described violated 72 principles.
>>>  Many of the violations were, by themselves, critical.
>
>
>
>  From what they know in the field of forecasting, the IPCC forecasts 
>are worthless.

	We have no idea of how many of those 'principles' are because 
the IPCC did not follow their definition of forecasting procedure (ie 
define alternative policies, assess the chance of policy 
implementation success, etc). In other words, without context and 
detail, knowing how many of their principles were violated is useless 
information to me, because I don't know how many of their principles 
are obviously foolish to apply to the issue they are discussing (and 
I have the clear evidence of earlier paragraphs that some of them 
clearly are). The IPCC report isn't a suitable basis, on its own, for 
management level planning of public policy. Yes. But its a perfectly 
good report for defining the extent of the problem for public policy 
to solve.
	I'd be less impressed with the IPCC report had it actually 
tried to do the things these self-congratulatory chuckle-heads say it 
should have. It would then clearly have been straying out of its area 
of expertise. As the forecasters clearly are, with gusto.

>  >> We have been unable to identify any scientific forecasts of
>>>  global warming. Claims that the Earth will get warmer have no more
>>>  credence than saying that it will get colder.
>
>
>This doesn't mean they believe the Earth will get colder. It just 
>means that if the only thing you have is those forecasts, being as
>they are, then you can't tell what it's going to do.

	Its still an extraordinarily silly thing to say. And based on 
silly reasoning.
	They are correct that relying on regression studies alone has 
problems because correlation doesn't imply a direct causal link, and 
that would be a problem if the IPCC relied on regression alone. But 
you know, they actually have a perfectly good scientific model of the 
causal role of CO2, and regression data is perfectly useful for 
determining the size of an effect for which you have a good 
scientific explanation, and also accompanying physics models to back 
it up.
	They are correct that simple models can be more useful than 
complex ones for some uses, particularly explanatory power. Here is 
the news flash - we have a simple model of how CO2 in the atmosphere 
works, its called the greenhouse effect.

>I've snipped the rest of your post.

	Sure.
	I'm all in favour of restricting the debate to the parts that 
interest us.

	Regards
		David