What's at stake is the degree of warming associated with our carbon dioxide emissions. In particular, to what extent the earth's many complex and not necessarily well understood feedback systems may mitigate (or exacerbate) temperature increases. I've long been skeptical of the more catastrophic scenarios, because all this carbon used to be in the atmosphere, which probably defines a ceiling on how bad it will get--a ceiling well below "WE'RE ALL GOING TO DIEEEEEEEE!!!" That said, I wouldn't really want to live in the Jurassic, and not just because I'm afraid of hundred-foot lizards. (for example, I am also afraid of the
Bearing this in mind, I think most people--including me--missed the biggest part of the climate emails story. Sexing up a graph is at best a misdemeanor. But a Declan McCullough story suggests a more disturbing possibility: the CRU's main computer model may be, to put it bluntly, complete rubbish.
As the leaked messages, and especially the HARRY_READ_ME.txt file, found their way around technical circles, two things happened: first, programmers unaffiliated with East Anglia started taking a close look at the quality of the CRU's code, and second, they began to feel sympathetic for anyone who had to spend three years (including working weekends) trying to make sense of code that appeared to be undocumented and buggy, while representing the core of CRU's climate model.
One programmer highlighted the error of relying on computer code that, if it generates an error message, continues as if nothing untoward ever occurred. Another debugged the code by pointing out why the output of a calculation that should always generate a positive number was incorrectly generating a negative one. A third concluded: "I feel for this guy. He's obviously spent years trying to get data from undocumented and completely messy sources."
Programmer-written comments inserted into CRU's Fortran code have drawn fire as well. The file briffa_sep98_d.pro says: "Apply a VERY ARTIFICAL correction for decline!!" and "APPLY ARTIFICIAL CORRECTION." Another, quantify_tsdcal.pro, says: "Low pass filtering at century and longer time scales never gets rid of the trend - so eventually I start to scale down the 120-yr low pass time series to mimic the effect of removing/adding longer time scales!"
The emails seem to describe a model which frequently breaks, and being constantly "tweaked" with manual interventions of dubious quality in order to make them fit the historical data. These stories suggest that the model, and the past manual interventions, are so poorly documented that CRU cannot now replicate its own past findings.
That is a big problem. The IPCC report, which is the most widely relied upon in policy circles, uses this model to estimate the costs of global warming. If those costs are unreliable, then any cost-benefit analysis is totally worthless.
Obviously, this also casts their reluctance to conform with FOI requests in a slightly different light.
That's not reason to abandon efforts to control our carbon emissions--as I say, they're still very likely to be problematic. But if the model turns out to be as bad as initial reports seem to imply, we should probably hold off on policy recommendations until we have a slightly better handle on the likely outcomes.
A 2006 Wall Street Journal article on the inner workings of the CRU (aka the Enron of science):
http://online.wsj.com/article/SB115283824428306460.html
...In addition to debunking the hockey stick, Mr. Wegman goes a step further in his report, attempting to answer why Mr. Mann's mistakes were not exposed by his fellow climatologists. Instead, it fell to two outsiders, Messrs. McIntyre and McKitrick, to uncover the errors.
Mr. Wegman brings to bear a technique called social-network analysis to examine the community of climate researchers. His conclusion is that the coterie of most frequently published climatologists is so insular and close-knit that no effective independent review of the work of Mr. Mann is likely. "As analyzed in our social network," Mr. Wegman writes, "there is a tightly knit group of individuals who passionately believe in their thesis." He continues: "However, our perception is that this group has a self-reinforcing feedback mechanism and, moreover, the work has been sufficiently politicized that they can hardly reassess their public positions without losing credibility."
In other words, climate research often more closely resembles a mutual-admiration society than a competitive and open-minded search for scientific knowledge. And Mr. Wegman's social-network graphs suggest that Mr. Mann himself -- and his hockey stick -- is at the center of that network....
So that doesn't mean I don't worry quite a lot.
Al Gore doesn't worry much either. He just bought a multimillion dollar condo in San Fran, eight blocks from the ocean, which is an odd investment if he thinks it's going to end up underwater. Kinda like the old joke about the doomsday cult leader who buys 30-year bonds.
Of course, it's also possible he saw the measurements showing sea level rise has flattened...
the CRU's main computer model may be, to put it bluntly, complete rubbish.
As a programmer, this code is just horrifying. Out in the private sector you'd be strung up for something this byzantine.
I've long been skeptical of the more catastrophic scenarios, because all this carbon used to be in the atmosphere, which probably defines a ceiling on how bad it will get
Yep, that includes some ice ages, too. As long as Antarctica stays over a pole things should be okay.
Academic code tends to be like that a lot of the time. It's typically a lot of spaghetti that never cares about error-handling or other niceties like documentation.
Most often, the people who wrote it aren't actually software developers, but rather people who learned a bit of programming to get at more interesting problems (which is fair enough), so I'm always a little afraid to peek at their code.
Yeah, I've worked on academic code myself. This sounds totally typical. Documentation and error handling take time and experience, which underpaid grad students don't have. We had certain experimental variables hard-coded in, so we would recompile every time we wanted to change something.
That's what happens when learning to do it right would take you six months, and doing it right would take you a month for each little bit of code, and you only want to be there for a few years anyway.
In a lot of cases, it's fine to have quickly developed code that just runs in a limited set of circumstances. I do it myself when I want to prototype something.
But once this becomes the basis for multi-trillion dollar decisions, I think the standard has to be a bit higher than "well it runs on my machine."
You'd think that when a computational model is the key element of a publication, review of the code used to implement the model would be absolutely required -- unless the code is a well known off-the-shelf product.
Look at the original SPICE code that served as the basis for the silicon industry is a perfect example of spaghetti code, but that was written well before modern software practices and in FORTRAN and DID have error checking.
I can live with spaghetti code from a Ph.D. candidate since after all it's only for one project. But spaghetti code without error checking from a continuing project on a project that will be used for determining how to spend trillions of dollars?!
I've been saying for years, though, that the simulation code was atrocious for the global warming fanatics. I saw the code they were using for their papers from another Ph.D. student years ago and freaked out. It is horrible, and if you dig into the code you'll see that they make predictions without accompanying error limits so that their predictions are completely meaningless, and without any error checking whatsoever.
And the scary part was that I found out he had gotten the base code from "major scientists in the field" was the most technically qualified of the code architects on that global warming project (shudder).
As a programmer, this code is just horrifying. Out in the private sector you'd be strung up for something this byzantine.
Or, you could just 'fix' the part where it keeps going like nothing happened following an error condition and make it have a meth trip instead, give it a snappy name such as Millenium Edition or Vista, and sell it for a hundred bucks a pop.
A hundred? Where'd you get the discount?
That only works when you control the distribution channels.
Ah, so it is just like the global warming theorists.
"...this code is just horrifying. Out in the private sector you'd be strung up for something this byzantine."
Not in my experience. The private sector is not idiot-proof; it just allows customers to avoid the more idiot-dominated vendors. In fact, many private-sector idiots are more diabolically energetic in their idiocy than anybody in the government outside of Congress.
The code sounds like a Special-Case Morass: Programmer doesn't understand the problem domain and gets it wrong to begin with. It goes back and forth between him and QA for six months, sprouting endless new special cases, each of which fixes 90% of one problem, creates 80% of another, and has a 10% chance of breaking the build. Eventually, the remaining cases where it fails are weird enough that nobody can define how they happen or reproduce them, and it's declared fit to release.
This happens because the programmer fails to understand the problem domain, and never learns.
But as others have observed, none of this necessarily tells us much about the kind of code we can expect top-notch scientists to write in the course of struggling to understand something that nobody fully understands yet. They're not professional programmers. It's a fundamentally different profession. I'd much rather hear from other scientists about this code.
However, those comments about "Apply a VERY ARTIFICAL correction for decline!!" and such, they sound like these guys are actually pretending to have already learned a lot more than they have. Unless they're meant to be jokes; many, many jokes do make it into comments in released code, and only a psychopath fires anybody over it.
When you're political allies are asserting that 'the debate is over' and 'the data are in', when potentially trillions of dollars of treasure and serious changes in the lifestyles of at least many millions of people are involved, jokes are not funny.
I had to write a bit of C and Assembly in engineering school, with only a minimum pre-req of C++ training, to solve problems in a couple labs and yes, the code was hideous and invariably contained a few caustic comment fields. However, my lab partner and I did not collect large sums of public grant money for this purpose. In fact, the only third party impacted by our inability to write and document good code was our long-suffering TA, who was likely deprived of $19.99 at the florist at least twice when our excursions deep into Spaghetti Land prevented him from returning home to his wife and daughter until sometime after 11pm.
The CRU model, as various ones keep noting, is one of THE premier climate models driving the literature and major government policies toward climate change. If its creators are not qualified to develop code that is competent and capable of producing repeatable results, then they ought to have hired programmers who could do it for them, else open-sourced the modeling project.
Meagan it is a question of priorities. I'll give you that CO2 above a certain level is bad for the environment and eventually us. Too bad that we don't have any clue what that level is. In the meantime we have contaminants in drinking water, air pollution, disease, hunger and the occasionally asteroid. What should we prioritize?
And we can't afford to do everything at once. In fact without a robust economy we really can't afford to do any of them.
How "Lomborg-ian" of you. :-)
CO2 poisoning is well documented and known. Submariners have dealt with it for about 100 years now, so there is plenty of science.
And Megan is right to point out that CO2, in and of itself, is something we should watch.
For example: if overall CO2 levels increase by a factor of ten (1,000%), city dwellers will have some issues since their ambient concentration will be more like 11 or 12 times higher. In meeting rooms, auditoriums, etc. you will easily be able to concentrate CO2 to the level where it makes you drowsy, or even a bit dizzy.
Of coures, to GET CO2 levels up that high we'd have to burn *all* of the remaining oil and coal and natural gas, because I don't think there's enough carbon sequestered in fossil fuels to increase CO2 levels that high unless we use'em all...
(i.e. "don't worry about it")
And plant growth would have to hit some other limiting factor so the greenery doesn't go -- ooooo! goodie! lots and lots of CO2 -- let's grow!
Yes CO2 poisoning is well documented, when found at much higher levels. The Minnesota Department of Health has limits set at "The DLI has set workplace safety standards of 10,000 ppm for an 8-hour period and 30,000 ppm for a 15 minute period. This means the average concentration over an 8-hour period should not exceed 10,000 ppm and the average concentration over a 15 minute period should not exceed 30,000 ppm." And, where do you get that "11 or 12 times higher", thin air. Modern commercial A/C systems have the ability to bring in fresh air and are usually set around 10%.
Moreover, when writing about the Medieval Warming Period and the Little Ice Age, CO2 Science Magazine (dated 27 March 2002), stated, "If there was truly a period near the beginning of the past millennium when temperatures were as warm as they are presently, for example, but when the atmosphere's CO2 concentration was about 90 ppm lower than it is today, and if that period was followed by a several-centuries-long cold period with essentially no decline in the air's CO2 content"
Maybe that little thing that comes up on your port side in the morning might have something to with it.
Make that starboard. I was in the Army, not the Navy.
MEGAN: I built programs like this, and they are crap.
Even if the program runs, they are TERRIBLE at modeling complex systems like climate.
And the emails reveal that the data they fed into it was crap as well. And they doctored the results. And they refused to share their data with anyone -- it was simply impossible to replicate any of their "results".
So, you have GIGO, and a crap process in the middle, and doctored GO, all with no peer review.
Add in active efforts to stifle dissent, and you have a Fraud. It is the opposite of Science. They should all be fired, before or after jail time.
Yep.
The bad code and bad data wouldn't be a big deal, except that they have the unmitigated temerity to then claim to be 90% certain they're right and demand we divert trillions of dollars to address the problem. And they refuse to release raw data and code.
I don't know in what world this is considered "science," but I'm pretty sure I wouldn't want to live there.
And they refuse to release raw data and code.
This to me is the biggest issue. If we're going to be making major decisions based on the output of these models, we damn well ought to be able to see the source code, or be able to verify their ability to accurately predict climactic behavior independently.
"And they refuse to release raw data and code."
There is no other issue.
Science demands that scientists who refuse to disclose their data and methodology be ridiculed as the charlatans they are.
The only morons who take the CRU model at its word are people with vested financial interests who are defrauding goverments for the grant money (which is a trivial feat).
No reputable scientists stand by the CRU models.
You'd think academics would support Open Source.
"You'd think academics would support Open Source."
Those who aren't scamming governments for millions in grant dollars do support open source.
Thieves and socialists, of course, cannot stand the sunlight.
You know you can see the source code right? Any number of the climate models are available for download.
Is there publicly available data to run through them?
Some GCM code is available, but the GISS algorithms were not being released last I checked.
Also, the CRU actually claims to have lost their raw data. That obviously looks especially bad now with all the deletions to avoid FOIA requests.
I don't see how it's possible to take anything from CRU seriously at all. They can't provide their data, the refuse to provide their models, and they've admittedly done extensive manual intervention to make things "work."
I don't think this invalidates the case for AGW, but it really kills the credibility of this particular institution.
everything we know about carbon dioxide indicates that it has a greenhouse effect
No, it doesn't. Greenhouses aren't hot because of CO2; they are hot because they are sealed containers with lots of dark surfaces. The earth and its atmosphere is NOT anything like a greenhouse.
Greenhouses are hot in part because glass is less transparent to heat radiation than light. And CO2 does the same thing in the atmosphere.
Please define "heat radiation" in this context. And how does it help for it to be more transparent to "heat radiation" than light?
We're talking the difference between IR (heat) and visible here.
Hell, gamma rays, technically, are "light".
Passive heat radiation is generally in the infrared range, while most of the sun's energy comes in at higher frequencies. The interior of the greenhouse, or the earth's surface, absorbs that incoming light energy and warms up, then begins radiating it back in the infrared range.
Glass, or carbon dioxide, can pass higher frequencies up into the low ultraviolet range with relative efficiency. The infrared frequencies, meanwhile, expend some of their energy heating the medium (glass, or carbon dioxide) and the rest tends to be reflected back into the 'box'. In order for heat to escape, the heated medium (glass, or atmosphere) must re-release it by the same process. So the heat does escape, but much more slowly.
You familiar with blackbody radiation? Basically, if we ignore reflected light, every object above absolute zero emits radiation. This radiation has a definite curve of how much is produced at what wavelengths, it looks sort of like a bell curve. The curve is dependent solely on the temperature of the object emitting the radiation - the hotter it is, the more radiation at any individual wavelength, and the more energetic the wavelength of the peak.
The Sun is about 6000 degrees, and so it emits radiation whose peak intensity is in the visible spectrum(hence why it's visible - there's more of it than anything else, so our eyes evolved to see it). The Earth is about 300(you need to use absolute degrees, not C/F), so it emits radiation whose peak intensity is in the infrared spectrum. Hence why night-vision goggles often work by converting IR to visible - the difference between a wall at room temperature and a human that's 20 degrees hotter becomes instantly visible, same as the difference between a campfire and a blowtorch.
In the absence of variance in the spectrum, the calculation of overall temperatures is easy - we get X amount of power from the sun, earth has a surface area of Y, so we tend to whatever temperature emits X/Y radiation per unit area. In the case of the Earth, that's way below zero - I don't remember the exact number, but it'd be polar conditions. However, we do have several important chemicals in our atmosphere that heat the planet up significantly - most notably, water and carbon dioxide. These chemicals have the property(same as the glass that greenhouses are built out of) that they are transparent to visible light(i.e., most of the Sun's radiation) but that they are more opaque to infrared(i.e., most of the Earth's radiation). As such, energy goes in quite well, but doesn't leave nearly as well - it gets stopped, some of it heats up the chemical in question, and a bunch of it is radiated back inside. The inside of the greenhouse thus heats up until it emits enough energy to radiate back out what it's getting in despite the glass(or carbon dioxide).
So yeah, while his language wasn't great, he was exactly right.
The important thing to remember is that radiated heat is proportional to the 4th degree of the absolute temperature. So radiational cooling at "normal temperatures" is only important when all other mechanisms (which tend to increase linearly with temperature) are suppressed.
Max, an increase from, say, 0 Celsius to 20 Celsius(273 to 293 Kelvin) would be an increase in radiation by a factor of (293/273)^4 = 1.33 - not overwhelming, but significant. However, that's less relevant to your point. Even if glass was special stuff built by Maxwell's Demon himself, and it stopped all radiation of all types from leaving, the greenhouse will still be cooled by conduction through the walls. The question is how high the temperature gets before conductive cooling is sufficient to negate the net radiative heating.
Of course, we're arguing here about parts of the analogy that do not apply to the actual debate at hand - the Earth is not cooled conductively, it is cooled pretty much solely by radiation. Still, conductive cooling does not negate the fact that greenhouse glass stores heat.
Joe is more right than wrong though. Convection is by far the biggest vector of heat loss at the temperatures we're talking about and the glass roof stops it very effectively. Radiation heat losses in an actual greenhouse can, most likely, be ignored (I imagine conductive losses through the walls and roof are bigger).
Now, in case of Earth, gravity takes care of convection. And vacuum takes care of conductive losses. So the two remaining mechanisms are dissipation from the far-flung tail of Maxwell (molecules with escape velocity or above) and the radiation, which, likely, dominates.
Greenhouses are hot in part because glass is less transparent to heat radiation than light. And CO2 does the same thing in the atmosphere.
Actually, it turns out that CO2 absorption spectrum considerably overlaps the H2O absorption spectrum. The atmosphere is, essentially, opaque to long infrared radiation even without CO2.
With all due respect I think you're both a bit off. Joe is off because when people refer to the "Greenhouse Effect" that does not necessarily imply that they mean that it works exactly as a greenhouse does (I would agree with him that it's a terrible metaphor for people who actually understand greenhouses, but that must amount to something like 1% of the population.) ech is off because Joe is correct that the degree to which glass is "less transparent" is pretty much irrelevant to the operation of greenhouses. Trust me on this.
My Mom wrote one of the definitive greenhouse books, and I spent a lot of my time as a kid working in, and occasionally putting up greenhouses. A lot of them were made of plastic which is not at all "less transparent to heat radiation" than is air. Polyethylene does very little by itself to stop radiative heat transfer.
Matters become more complicated because water tends to condense on the inside surfaces of a greenhouse at night, and that does have some effect on radiative heat transfer, at the time you most need it. But that's not the main principle on which greenhouses work- they work because they stop convective cooling. It should be noted that climate models cannot even begin to model convective cooling, though without it the earth would certainly be unliveably hot (given, of course, the radiative forcings provided by greenhouse gases, without which the Earth would be unliveably cold.)
You really need to look a bit, just a bit, at the science. If it wasn't for the greenhouse effect the surface of the Earth would be much colder. See http://en.wikipedia.org/wiki/Snowball_Earth for discussion of an extreme (and disputed) case.
I think you're missing the point. No one who knows anything about the subject denies that there is a "Greenhouse Effect" in the sense that the Earth's atmosphere contains gases that inhibit radiative transfer of heat to space. We all know that the Earth would be very cold if not for that. No one disputes that. Joe is just saying that that mechanism should not be called a "greenhouse effect" because that's not how greenhouses work. He's correct about that, but I can see how someone who didn't know much (or care much) about greenhouses might find that needlessly pedantic.
I was wondering if you were going to come back to this. As the story has developed, the implications of all this have grown in scope significantly.
The CRU's HadCru temperature records is a starting and reference point used in many derivative and supporting climate research. If HadCru becomes worthless, it casts serious questions on countless other studies.
"The CRU's HadCru temperature records is a starting and reference point used in many derivative and supporting climate research. If HadCru becomes worthless, it casts serious questions on countless other studies."
This has been the problem all along. Garbage in, garbage out.
Almost every other climate model relies on this single source of crap data. That's why CRU refused to release the data and model under threat of fines and jail even. They knew it was crap data all along.
These people are criminals who have been defrauding taxpayers of grant money for decades. Now their little secret is out and prosecutors should begin their interrogations and attempt to get the grant money back.
"Almost every other climate model relies on this single source of crap data."
No that's not true.
"No, that's not true."
Oh, OK, we'll just take your word for it then, since you have stated your case so cogently.
Well, Despain is half right, which is more than usual.
Once Steve McIntyre demolished Mann's Hockey Stick, Briffa and Osborn created a new one that some people probably relied on before McIntyre trashed it, too.
Is there somewhere that large volumes of climate data is made publicly available? I'm asking w/o snark; I think it would be a great dataset for lots of types of analysis.
"Almost every other climate model relies on this single source of crap data." No that's not true.
CRU was, at least, the most often cited by the IPCC.
http://www.theregister.co.uk/2009/08/13/cru_missing/
They did not release temp processing code (which is why it had to be leaked) and claim to have lost the raw data. This is hardly a good basis for any kind of science, let alone the foundation for policies costing trillions of dollars.
I offered the same level of proof you provided in your original post.
TW - Yes but the data sets are large. If spend about 5 minutes at Google you should be able to find various data sets. Which ones are you interested in or what type of data?
Either my google-fu isn't very good, or the interwebs are all clogged by commentary on the topic. I'd be most interested in point measurements of various factors (temperature, precipitation, etc) with associated time and location.
I've got time and a dedicated ftp server for moving large datasets around.
I'd like to see the CRUs raw data... which they say is lost.
CRU's data has licensing issues as well. You should probably use GISS
Here you go.
http://data.giss.nasa.gov/
Yes, the GISS ModelE simulations are available. But the GISS data is hopelessly corrupted, and as best I can tell the algorithms they use to adjust temps are not available.
I also have real problems with using the "licensing" excuse while using the data to advocate for policies costing the public trillions of dollars. Show your work or go home.
the same folks who wanted to look at HadCru raw data, algorithms and code have the same request and suspicions regarding NASA's GISS stuff.
GISS is run by James Hansen -- a scientist who is equal part strident AGW political activist.
You'll see Hanson in the CC line of many of the related e-mails.
From the "debugging" post:
That tends to happen when your numbers are imaginary...Heh. Of course it really comes from an overflow error.
Oh boy. I wonder if anyone is using i in the climate model, and if some uninformed politician/pundit/commenter is going to go off on "proof" that scientists use imaginary numbers.
It's unlikely to be called "i" in Fortran code though :-D
True. In C/C++ programs I've worked with the struct/class is called "Imaginary" or similar, with the instance variable being "imag". With a comment "//Imaginary number." because I rarely use them.
It'd be std::complex<double> or similar in C++.
And, IIRC (yes, IRC -- I cheated and looked) in Fortran-77 and above there is a language-level literal for the complex numbers that looks like (12345,6789) which means 1235 + 6789i.
The point though is that "I" tends to be a loop variable in Fortran programs :-)
Variables starting with I-N are I-Nteger numbers IIRC from my FORTRAN class many years ago.
The point goes a bit beyond that Max. It's been a long time since I programmed in Fortran (thank you god) but "old Fortran" has rather byzantine rules about capitalization (and which variables can appear in which columns) that date from the time when it was input with punch cards. Fortran still has a lot of cruft left over from that time period, but it also has some of the most efficient libraries for scientific mathematics (matrix operations in particular) that exist. I'd argue that since modern machines are roughly over 9000 times as good as the older ones we ought to ditch Fortran, but... it's still widely used in certain scientific applications. I have thankfully forgotten all the details.
Anyway I think it was Tony Hoare (my Hoare number is 1, btw, using zero indexing of course) who said "I don't know what the language of the year 2000 will look like, but I know it will be called Fortran."
That was a classic.
This is one of the places where I'm with Jim Manzi--all of the code generating these models should be open-sourced, and snapshots of the code should be placed into escrow at regular intervals so as to test the accuracy of the predictions that the models make.
I suspect that the accuracy of the models would be discovered to be appalling.
Many of them are. CRU's are problematic for various licensing reasons.
"CRU's are problematic for various licensing reasons."
coughbullshitcough
CRU's are problematic for being fraudulent, corrupt and ineptly programmed. EULA's are just the latest lame excuse to keep the sunshine out.
Doesn't matter now, though. All their code are belong to us.
It is in the public domain and being ripped to fucking shreds by computer scientists the world over.
We're routing around the fraudulent scientist nodes.
Sorry but just because you don't take contracts seriously - other people do.
"Doesn't matter now, though. All their code are belong to us.
It is in the public domain and being ripped to fucking shreds by computer scientists the world over."
The code was pretty much always available. It's the data sources that are problematic.
The thing is that there's nothing in the emails or the released code that serious computer scientists didn't already know. The emails are useful in that they show dishonesty in ways that people are good at understanding- collusion, ostracism of people you disagree with, suppression of results that disagree with yours.
But they show nothing that wasn't obvious to any serious computer scientist before their release. In fact, they show less than is obvious to any competent computer scientist. The claims that the modelers have made for years should have been far more damaging to their credibility then the release of these emails is. But the world is mostly made of people who understand that collusion and fraud are bad, but who don't understand computation very well. If journalists understood computation they would have discounted these guys in the first place, and saved us all a lot of time, trouble, and money.
>>Sorry but just because you don't take contracts seriously - other people do.
They are welcome to take contracts seriously. I, on the other hand, take peer review seriously. Their contractual inability to release their data should preclude publication of their results in peer reviewed journals and deprive their results of the status granted to such publications.
It would really surprise me if the various licensors couldn't be brought around to the point of view that they should license their code and open it up. Given that multi-trillion dollar decisions are being made on the topic, it seems like paying them virtually whatever they want would be an acceptable part of due diligence.
If they don't want to open it up on principle, well, I think that's problematic in terms of taking their conclusions too seriously.
Well the problem is that many of the countries donating the data also resell it. I guess if someone was serious they could license the data from the contributing countries. That would cost some cash though.
That would cost some cash though.
Seems like it would be worth making the investment, given the kinds of decisions that are being made on the basis of the data. I know I would be a *lot* more likely to actively support legislation if I knew that all the inputs to that legislation were public and and been tested and retested.
"But if the model turns out to be as bad as initial reports seem to imply, we should probably hold off on policy recommendations until we have a slightly better handle on the likely outcomes."
Yes, that would be the sensible thing to do, if our academy, elected leaders, and media cared about good science. But they don't. The scientific process is no longer relevant here. The only thing that matters is the political process. Too many interest groups have far too much invested in the narrative to stop the politics of AGW now. If the science isn't as convincing as portrayed (and thus the policy prescriptions not as effective, if effective at all), too bad. That's why the CRU scientists did what they did. They knew their data, their procedures, etc., did not conform with the emerging narrative, so they fudged.
Bad code is bad, but undocumented code is just as bad. Bad undocumented code? Oy vey.
Years ago, when I was just starting out as a programmer, I had to debug some C code in an industrial automation application that ran on a i386 DOS machine. The hardware allowed 24 pallets of parts, the input screens allowed you to set it up to handle 24 pallets of parts, but if you set it up for 24 pallets the machine ran right into its overtravel switches at full speed. In fact, more than 16 pallets caused this.
Deep inside the code was a section that, when printed out, took up two pages. A while() loop (for the 1 to 24 pallets) wrapping two for() loops (x-axis and y-axis) which themselves wrapped an if that had the pow function as the condition. After tracing it out by hand I determined that the if condition was checking to see if a bit in a 16 bit int was set. Every piece of code that handled positioning (including virtual positioning within arrays of output data) had this nice little bomb in it. Because, you see, if you try to set 24 bits in a 16 bit int it just wraps around and resets the first 8 bits.
Comments? Well. At the very top of the loop was the following:
//Why did I do this?
I have been a demon for commenting ever since.
Once, in fixing some code written by a very poor contractor, spent 20 minutes staring at a method before simply documenting it "throws a NullPointerException".
I didn't delete it until the next week for fear of the the avalanche of compiler errors that would be created if it were missing.
Hmm- please allow me my own little code saying: the apprentie doesn't comment his code. The Journeyman comments his code extensively. The Master doesn'rt comment his code ;)
The point is that comments should actually be used sparingly, but code should be written so that that is not a problem. I don't know if you've ever inherited a large codebase that contained a lot of comments that were _no longer correct_. I prefer to write code that documents itself in the small and leave explicit comments for larger architectural issues.
I truly don't understand this.
There are, what, 3-5 major institutions of mass-data and analysis that are driving the climate modeling behind AGW?
And this topic is being discussed to the degree that countries are making wide-ranging policy decisions.
And yet the raw data and statistical modeling has not been made public by these institutions (which, as far as I can tell, are publicly funded places).
How on Earth is this considered any form of settled science?
Settled science is when research is done with sound data from many sources and made available, modeled to consistently yield the same or very very similar result, and that both the data and methodology of the modeling is reviewed far and wide and the only questions that arise are minor nit-picking which goes on for years (as any theory will experience).
This whole topic seems to have none of those: the models predictions aren't proving true over a decade period (and yet make predictions of what will happen over the next 5-10 decades), the data is widely unavailable for review, the data comes from narrow sources of questionable accuracy due to improvements in technology and the short window of testing, the modeling systems used tend to not be shared....
and at the same time, there are lots of reputable people with serious questions about the veracity of the data.
This is not settled science. And, for those who think that it's only propaganda causing people to doubt AGW, please realize that propaganda has been around since the dawn of time. We'll settled lots of scores with propaganda in the mix. The way to fight propaganda is to have lots of people on your side and be very transaparent with your information.
Just my .02
It's not settled science, and never has been. But it's been presented as settled science by the Left and various corporate, political, media, and governmental interests. The motive is simple: M-O-N-E-Y. Along with power.
"That's not reason to abandon efforts to control our carbon emissions"
I don't think that the debate has ever been about whether we should take some actions to try to limit carbon emissions (and pollution in general).
The debate is over whether to 1) focus on moderate, common-sense steps such as more nuclear power while continuing (non-government-directed) research on alternative energy sources, or 2) take drastic steps that will dramatically reduce global economic growth and force more and more people to live the lifestyles that many liberals want to force on everyone anyway.
They rigged the data and models to try to trick us into taking drastic steps, because a more scientific approach would have indicated a more moderate response.
also why concepts like carbon sequestration and mitigation approaches are ignored by the environmental left--
central to their belief system is the idea that society must pay for their destruction of the planet.
Yes yes yes.
The technology isn't there yet. It probably will come, but to suggest something MUST BE DONE NOW IN A HURRY OR WE WILL ALL DIE dooms us to making poor decisions that may in fact make the problem worse.
Basing everything on borderline fraud and misrepresentation is a good first step to making the problem worse.
Derek
If a scientific hypothesis cannot be confirmed, it is unproven. It increasingly appears the CRU team couldn't even generate accurate "predictions" of past events, let alone future ones.
If HadCru becomes worthless, it casts serious questions on countless other studies.
One crucial question is, how much of current climate-change science consists of the CRU's work, or is wholly dependent on it? How much must be substantially revised if CRU data is unreliable?
I know it doesn't break down this neatly, but as a first approximation, does this mean 25% of the relevant science is worthless? 10%? 75%? I'd really appreciate a response from anyone well versed in the area.
I'd love to see the forum at The Daily WTF do a once over on this code. Hilarity would most definitely ensue, except for the trillions of dollars that are being bet on crap code.
Why don't we talk about the link between pollution and disease, as an argument for cleaning up the environment, instead of debating global-warming? Because global warming science can be debated ad nauseam. Because it's easily politicized.
Because the global warming debate is a ruse.
If you're asthmatic you can relate more to the thick smog layer hovering over your backyard, then you can to planet warming associated with carbon dioxide emissions. Yet we never talk about that smog layer, or your asthma. Purposefully, I believe.
Megan,
The more trenchant criticisms of the global warming religious order have for years (Richard Lindzen in particular) focussed on the complete inadequacy of the computer modelling that was being done. This line of attack seemed to be generally ignored by the faithful. This should come as no surprise to anyone who has been paying attention.
Because global warming science can be debated ad nauseam. Because it's easily politicized.
Ah yes, the old "who cares if its true because it leads us to socialist policies we need to adopt anyway" defense.
"I said before that I don't think the emails refuted the notion that AGW is real, and happening."
This isn't the issue as everyone knows. A process failure doesn't prove or disprove a scientific hypothesis. It does however mean that any "evidence" generated through the process is faulty and cannot be relied. So this process failure shows the evidence used to support the alarmists' assertions is woefully inadequate and that said alarmists are colluding to keep pertinent facts from public knowledge. Maybe New York schools need to stop teaching political activism and focus on the scientific process and basic logic.
Non-alarmists have have been asking questions for decades without meaningful response from the alarmist community. They now know why as does everyone not making a Herculean effort not to understand, which apparently includes you.
On the other hand if we eliminate your strawman to focus on the issue we arrive at this: do these emails prove that leading AGW alarmists are actively and intentionally impeding scientific inquiry? The correspondence proves that these so called scientists are not engaging in science. They are political activists in white lab coats.
While you're right you fail to identify the most important strawman here. I think I could be fairly classed as a skeptic, but I'm quite willing to posit that humans have changed the climate. In fact I think you'd have to be an idiot to think otherwise. And, if pressed, I'd guess that increased CO2 levels will, absent other factors, lead to warming.
But we know how much warming increased CO2 will lead to, by itself, and that degree of warming is nothing to worry about- we ought to welcome it. So the argument is _100%_ about whether the feedbacks are negative or positive, or neutral.
Any argument about warming that fails to acknowledge that should be dismissed out of hand. I wonder how many journalists understand that at the levels we're talking about increased CO2 cannot possibly warm the Earth much by itself, and that even the most hysterical advocate-scientists don't dispute that.
Does anyone know if the dataset is included in the material that was taken? Having that data in the public domain would be fabulous.
Much of the data has been eaten by a graduate student's dog.
It was "deleted" because they "ran out of disk space."
Yea, right.
Much of the code in the academic world tends to be written by grad students that have taken a class in programming and get told to write it. As a result, the code tends to be really, really low quality. I've seen some if it. It's not pretty.
I was offered a graduate school position in a branch of physics at one school simply because the professor needed a programmer on the team. I had taken 9 hrs. of programming and worked summers as a programmer for a Fortune 500 company and knew how to write straightforward, well documented code. I had also written some spectral analysis programs for my undergraduate research that they liked. It was a sweet gig - full scholarship with a generous stipend, but it was a field (terrestrial magnetic field interactions with solar output) that I was uninterested in. All were in Fortran like much of the code in the CRU case.
So, we have the world's economy being reshaped politicos that can't spell Fortran based on output from code written by a series of amateurs in a fashion that shows that they didn't pay attention in whatever programming class they took. It's like having the restructuring of the banks that got TARP money done by someone that had taken ECON 101 and slept through most of it.
I work in aerospace now and all our critical modeling for aerodynamics is done using professionally written code that is tested out the wazoo, has been checked against reality, and is known to work. And we only have billions at stake, not trillions.
Much of the code in the academic world tends to be written by grad students that have taken a class in programming and get told to write it.
This is totally untrue. I never took a class in programming before writing my crappy undocumented code.
I work in aerospace now and all our critical modeling for aerodynamics is done using professionally written code that is tested out the wazoo, has been checked against reality, and is known to work. And we only have billions at stake, not trillions.
This whole situation is positively Hansonian. The people who write code in industry have an incentive to develop true beliefs because they would get fired or even arrested if they wrote fraudulent code. That is not true of academics. Thus they have the luxury of using their beliefs to feel good about themselves.
"The people who write code in industry have an incentive to develop true beliefs because they would get fired or even arrested if they wrote fraudulent code."
In the universities, the reverse is true.
You don't get millions and millions of dollars in government grants to study global warming if your climate model shows cooling and nothing bad happening.
The grant money incentive guarantees all climate models will show catastrophe ... always just around the corner if we don't act soon.
Ah the old chestnut that they are doing it for the grant money. That's one of the more ridiculous ideas I have ever heard. Graduate students are paid peanuts compared to similar degreed jobs in the private sector. At least the graduate students have a job Mover.
It's not the grads doing it, it's the profs (who are paid at least cashews, if not macadamias). Grads just do what the profs tell them.
Graduate students are motivated by (1) the greater intellectual autonomy that comes in academia, and (2) the greater social status that comes from academia, and (3) the greater ideological autonomy (that is, to express one's ideological beliefs provided that they are orthodox) that comes from academia.
As someone clinging somewhere in the middle of the academic ladder and daily looking around wondering why? why am I clinging to this ladder? I totally understand the near-crushing desperation to secure grant money. Grant money requires papers, papers require data, and data are sometimes slow in coming. And nothing is so hard to let go of as preliminary data that doesn't hold up in the long run. BUT I've certainly never faked anything, and I've never not been able to find primary data or reconstruct how I arrived at a particular graph, figure, or statistical analysis!!! The apparent mismanagement of the raw datasets is what is so unbelievable about all this.
Grad students probably worry less about grant funding, but their time pressure comes from trying to collect enough data to finish the thesis to graduate. Because who wants to spend any more time than needed making peanuts, when everyone around you is making more in the private sector? I didn't worry about grants in grad school, but I did crank out data as fast as I could to get through as fast as I could. Took more shortcuts with some experiments than I should have (then I couldn't use the results, had to do them again). Of course, in climate science, you can't exactly "run the experiment" again...
Any graduate student who is trying to crank out crap so he can finish his thesis is not going to be of much worth to anyone.
The 25% of graduate students at the top who aren't wasting everyone's time and money are trying to figure out something new, because they know that good ideas and rigorous experiments are going to pay off for them in the medium and long run. A top graduate student knows he ain't gonna get a great job with crap research, even if he finishes early.
I'm happy to consider the alternate position here. If you are a grad student (or professor) who really can see that GW is bs, then there's a good chance that a number of others will see it too. Sure a number of these might go along with the prevailing idiocy, but many of these rebels/skeptics are going to revel in going against the general consensus.
Steve--just to clarify, I didn't crank out BAD data because of time pressures, nor did I mean to imply that lots of bad data is better than no data. Bad data is worse than no data because you end up wandering down all sorts of rabbit holes that go nowhere. I just meant that while it is true that PIs in academics are under enormous pressure to generate data to receive grant money, grad students are under pressure to generate data to complete their theses and move on. The best grad students I have known--in addition to working on new, good ideas and performing rigorously controlled experiments--also work 60+ hrs/week cranking out good data to test new hypotheses.
Fraudulent is probably too harsh a term. All code has errors in it. Amaturish is more accurate. Whether the results were fraudulent is a different story, but I don't believe the code was intended to produce particular results.
I think some of you are overstating the quality of commercial code a little bit. There's some terrible dreck out there being sold for money too. Oddly enough, banks are major culprits for having incredibly poor quality code in their internal trading systems, although I think the root causes for that are different to the root causes here.
I don't see much going on with CRU stuff that I haven't also seen happen in commercial projects of an equivalent level of intellectual difficulty - they have a lot of problems with poor data and a lot of problems with programmers not completely understanding the problem domain. And this is understandable because the amount of data is vast, and the problem domain is genuinely very complex.
There's really not much here that someone with experience on large, complex software projects in fields tougher than database backed web site development (which is after all 90% of the software industry), couldn't have guessed at. It just confirms the obvious - computer models are just very complex mathemetical models, as such they just reflect the assumptions of the people who developed them. If the assumptions aren't made clear, the model is pretyt useless.
Actually, that would have been an improvement.
It would indeed. I expect it's going to happen though, unless it gets accidentally deleted on the way. No one will trust this stuff any more without the data and algorithms. Which can only be good.
"One crucial question is, how much of current climate-change science consists of the CRU's work, or is wholly dependent on it? How much must be substantially revised if CRU data is unreliable?" I'd love to know that too.
Well CRU is a piece but's not like everyone was dependent on it. Additionally I find it amusing that of the 160 GBs stolen - 13 emails were selectively leaked.
What nonsense are you talking about with the "13 selectively leaked" garbage?
Here's all of them: http://www.eastangliaemails.com/index.php
Brian Despain is a leaked email denier.
Pay no attention to him.
Thanx for posting. Not many places are posting all the emails.
Brian,
I've rarely seen such blunt dishonesty like that. Here's your quotes, 24 minutes apart:
"Additionally I find it amusing that of the 160 GBs stolen - 13 emails were selectively leaked. "
"Thanx for posting. Not many places are posting all the emails."
So you admit to knowing that there were alot more than 13 e-mails leaked, and yet you originally use that as if it's being "selective".
Then, 24 minutes later after someone posts the full list, you admit to knowing that "Not many places are posting all the e-mails"
Wow.
It's not dishonesty but a quickly typed response. Real Climate didn't post any of the emails so I had no idea that there was more than 13. When I said all the emails, I was under the impression that only a few had been posted and more was out there and hadn't been posted. Of course more than 13 emails were leaked since the total file size was 160 GB - I just thought they weren't all posted
You really should assume someone is lying and I can assure you that's not the case here.
Real Climate didn't post any of the emails so I had no idea that there was more than 13.
Heh. This is sort of emblematic of the problem with AGW.
If you limit your sources of information to people who agree with you, you're bound to come to bad conclusions.
In fairness, it's possible he had only visited sites which were posting a baker's dozen of the most critical emails, and had not yet seen the full list disclosed.
I do read other sources. A number of blogs only listed the problematic emails as opposed to all of them. Which of course is understandable given the size of the archive which further googling has revealed to be 61Megs in size.
I do read other sources. A number of blogs only listed the problematic emails as opposed to all of them. Which of course is understandable given the size of the archive which further googling has revealed to be 61Megs in size.
Doesn't he know that there is consensus that the emails were leaked?
"Doesn't he know that there is consensus that the emails were leaked?"
That is settled, of course. There can be no debate.
Brian got himself caught in a lie ... attempting to frame the issue as one of context by suggesting that the emails have been taken out of context to paint a false light. He knows that avenue to be untrue, but he tried that angle anyway.
Then, when someone fact-checks him right up the keister and demonstrates that no, thousands of damning emails have all been released along with thousands of lines of faked data and poorly constructed computer model code, he backtracks quickly on his spurious allegations made just moments before.
Folks like him will do anything ... say anything ... to keep the lie going. Too much of their money is invested; too much of their prestige is at stake.
But the cat is out of the bag as the comment sections of every major website prove by having a field day with the cognitive dissonance of the liberal socialists.
Sorry - I was commenting on TW Andrews post, but ech got there first.
What a bunch of fair-weather (pun intended!) friends we all are!
We're going to let little things like an unexplained decade of cooling in the midst of steadily rising CO2 and the exposure of one of the main climate change research institutes as an utter fraud dissuade us from our belief in the urgent need for drastic change? I should hope not!
Look, I think we can all agree that regardless of what 'the facts' or 'science' or 'every day experience' might tell us, it is of the utmost importance that we immediately begin living our lives according to the sacred principles of carbon neutrality. That's what's best for Gaia and if we disobey she will surely punish us mightily. Al Gore has foreseen it!
Frankly, I feel somewhat dirty even listening to all of the heretics & heathens in here unleashing their vile blasphemies against mother Gaia. I'm going to have to do extra dumpster-diving and perform an additional protest against the eating of meat in penance for even drifting in.
Remember: if we want to avoid the all-destroying flood & the immolating heat, we must follow those who have seen the truth, for truly their way is salvation.
"unexplained decade of cooling" Really SO 2009 wasn't the 5th hottest year on record? Oh it was. You think the climate should linearly rise in temperature?
People who believe that there is no man caused global warming, at all, have trouble with operations in the complex plane. Especially as described by Lorenz and Mandelbrot.
No, we have problem with clinging to models whose predictions have been falsified.
But luckily there is more junky code and bogus data to be written. I'm sure you'll get CRU 2.0 properly backdated in no time.
Reply to Hagios:
"Whoosh" as the joke goes right over your head.
But there's a lot of math/comp sci geekery going on in the comments to this post. People without calculus should Beware.
Wiredog,
What are you talking about? Lorentz and Mandelbrot, the climate is a chaotic system. There was nothing mathematically sophisticated about your comment. My objection remains correct: reality has consistently falsified the predictions in the models.
You think the climate should linearly rise in temperature?
No, I think that anyone who wants me to believe he can predict the future should have a track record of successfully predicting the future.
Or at bare minimum, 'not have a track record of failing to predict the future'.
I enjoy Megan's blog because it has smart progressives, but this thread is like clubbing baby seals. On one hand you feel guilty. But on the other hand it is deliciously fun!
I'd be happy if they could predict the present using past data.
Climate models have. Which model do you think is the least accurate?
Which model do you think is the least accurate?
I think Hansen's 1988 A model wins that one.
Any model failing to predict the last 10 years--or, at a minimum, have error bars sufficiently large to cover them--is a failure.
They have? Which ones? And I assume that means they all agreed on the present climate so they had the same results.
this is where climate debate goes off the rails --
both are correct. Over the last ten years, it has been hotter than the long term "average" (BTW- the average is usually based on HadCru historical temps!!) However, the "excess" over average over the last ten years is getting smaller -- i.e. the temperature is moving back towards the average instead of increasing.
What is relevant is this: 20 years ago James Hansen kicked off this whole issue with a temperature prediction model showing that as CO2 was added to the atmosphere, the temperature would increase.
Today, after a reasonable "run" period of the model -- its off. Actual temps are more than two standard deviations away (lower) from what was predicted. That is statistically significant, and calls into question just how good this model is.
lso --FWIW -- the CRU emails have an exchange that goes something like "temperatures are lower and its a travesty that we cant explain it". The above is what that email refers to.
That was sarcasm, as if an initial model that doesn't perfectly accurately predict the future should doom the entire discipline.
It's not surprising that the first model, 20 years ago, hasn't worked all that well. The Wright Brothers first airplane didn't work all that well.
Subsequent models have worked much better. But you don't hear that from the lying denialists.
Actualy, what is happening in the last 10 years is that the rate of warming is decreasing. But it is still warming, not moving back to the average.
It's not surprising that the first model, 20 years ago, hasn't worked all that well.
At what point between a model that doesn't work all that well and today did this become so utterly settled and perfectly undeniable that any further discussion is evidence of lying bad faith denialism?
We're talking about a system vastly more complex than, say, an atom, and also one which does not lend itself to controlled experiments, but considerably more than 20 years passed between the positing of the idea of an atom and our current understanding of them.
I don't think you can make excuses for 20-year-old models as "not surprisingly" bad and still claim that the current consensus is utterly unshakable and will never change.
The Wright Brothers were not the first persons to attempt self-powered flight. They are remembered because they were the first persons to get an aircraft off the ground long enough to show that they were actually on to something with their design principles. I'm not sure you've quite assimilated the lesson in that if you intend to defend a lousy climate model by this means.
I suggest a quick visit to www.surfacestations.org for your edification and amusement.
There, you will find the results of independent surveys of ~80% of the US temperature measuring stations, which grade them using the criteria developed by the National Climactic Data Center. These surveys determined that the average station is subject to errors equal to or greater than 2C because of location, condition or both.
These temperature measurements are the raw data collected at the front end of the CRU and GISS processes. They are then "massaged" and "adjusted", thus becoming "undata" before being fed into the climate models. These temperature measurements are also the basis of the temperature anomalies reported by both CRU and GISS, to two decimal place "accuracy" no less. (I was surprised not to see the name Rumpelstiltskin among the authors of the CRU e-mails. :-) )
It is long past time for this entire process to go "open kimono". Then, cooler heads can decide whether to invest several $ millions to collect decent temperature data before investing several tens of $ trillions remaking the US energy infrastructure. Seems only fittin'.
Is there any actual evidence that atmospheric carbon dioxide increases generate a proportionate increase in temperature (as opposed, say, to an asymptotic approach to some upper limit set by other factors, or even a paradoxical movement in the other direction)? Or that other factors, such as the sulfur-based effluent emissions recently identified as having a "global cooling" effect, or natural climate change cycles, are not overwhelming it? Not plausible speculation based on theoretical models, but actual tested hypotheses whose support is not contaminated by the CRU's questionable methods?
I would be happy to accept AGW as the premise for a science fiction novel. For a policy that's going to cost trillions of dollars and massively alter people's lives, I want more than plausibility. What's the actual evidence?
Is there any actual evidence that atmospheric carbon dioxide increases generate a proportionate increase in temperature
Most everyone agrees a CO2 doubling leads directly to about 1 degree rise in temp. The debate is over the feedbacks. AGWers claim there are huge positive feedbacks, meaning a doubling in CO2 should lead to many degrees of warming. The evidence says feedbacks are probably negative, meaning the net warming should be less than one degree.
Or that other factors, such as the sulfur-based effluent emissions recently identified as having a "global cooling" effect, or natural climate change cycles, are not overwhelming it
That's why you generally don't see rising temperatures following a rise in CO2 in the temp record; CO2's effect is very weak and tends to vanish in the noise. Over at Watts' someone did a fairly sophisticated study of this.
I don't think anyone has claimed it's proportionate, just that there's a positive correlation. This is why they keep investing so much time, effort, and bad code into their models - they want to see just how it correlates.
Ed, what you are saying isn't entirely convincing. The individual measurement could be imprecise, and yet a composite of them might have much higher precision. For instance, for normally distributed data, the standard deviation of the mean goes like the inverse square root of the number of points. So if the initial data was +-2 degrees, and you had ten thousand data points, the error of the composite mean might be 2/sqrt(10000)=0.02.
Not saying that's happening here; don't know anything about it. But it's not necessarily a problem that the individual measurements are imprecise, as long as they're not biased.
That only works if the individual measurements being averaged are measuring the same thing. We're talking about different weather stations at different locations, meaning there's nothing to average out the error.
Also, there's uncontrolled systematic bias involved.
Bias, regrettably, is the problem. Location of measurement stations at airports and sewage treatment plants, on the roofs of parking structures and firehouses, adjacent to concrete and blacktop pads, etc. produces a positive bias to the measured data.
The NCDC recommendations are intended to avoid this bias. The obvious deviations of most of the surveyed sites from these recommendations, as documented surfacestations.org, can reasonably be expected to positively bias the data.
I am not questioning the precision of the individual measurements. I am questioning their representation of geographic temperatures they are supposedly intended to measure. One of the most pervasive concerns regarding these measurements is the Urban Heat Island (UHI)effect. There is little doubt that measurement stations located on building roofs, at airports, or at sewage treatment facilities would be positively impacted by UHI effects.
This only applies for random errors. If there are systematic biases in the measurements they won't necessarily be averaged out over large aggregations.
I agree about the bias. But as to them being different locations, I assume they are creating some kind of overall regional or planetary mean temperature from the different locations.
There's no reason to think that the errors from different locations are normally distributed, so you can't use that assumption to calculate the error in the mean.
I still don't--the fact is, everything we know about carbon dioxide indicates that it has a greenhouse effect, because it is more efficient at passing sunlight through to the earth, than at allowing that energy to reradiate back into space as heat.
Even the AGW-proponents don't believe that. Sure, C02 has a warming effect but it decreases logarithmically with concentration. Our best understanding suggests that carbon alone can't cause significant global warming. That's why AGW-proponents champion a "second shooter" theory. C02 regulates some other, as yet unidentified, warming agent.
For those of you keeping score at home: that's epicycle #1.
Yeah, that's the big problem. They're claiming this incredibly stable system can be upset by a change in a relatively unimportant trace gas.
Every absorption process in chemistry decreases logarithmically - this is not news. That said, the worst-case Al Gore scenarios are about a 2-3% change in surface temperature. Just because something decreases logarithmically does not mean that it can't provoke something that really isn't that huge a relative change.
Please, if you're going to attack AGW theory, stop making me look bad when you do. I really don't want to have to drag out my old chemistry textbooks to do first-order approximations in order to argue against someone on my own side.
Hmmm, someone predicted that climategate would be a face-saving way to hop off the AGW bandwagon. I wonder if we're starting to see the first signs? Megan is still on, but she's stood up and moved closer to the doors.
You know the best part of being a scam artist: People will almost never admit they've been had - even to themselves.
It's just human nature.
Expect AGW to just fade away ... much like the "Coming Ice Age" scare stories featured in Newsweek 30 years ago, or the Swine Flu Pandemic scare stories just months ago.
You're almost there. People don't like to admit to being had, but they will do so if they can use "I was lied to" as an excuse.
You heard it in 2001-2002 about Bush, you'll probably hear it more next year about Obama, and it's possible that this particular incident could change beliefs about AGW.
One point that has been repeatedly raised by AGW apologists is how much increase “we” are willing to accept since it increases warming. But what if more CO2 will not result in increased warming? I can’t remember the source, it was a book I read in the early to mid 1990s, but the author said that “greenhouse gases” work by absorbing heat radiated by the earth rather than allowing it to reach space, and that different gases absorbed different frequencies of IR radiation. He claimed that the radiation absorbed by CO2 was already almost completely absorbed and no more warming from that source could be anticipated. Which seems to fit with the apparent end of the warming trend in the last decade. Anyone have any information on this? I hadn’t mentioned it before, because I cannot recover the source, and have never seen a similar argument anywhere else. I posted a slightly different version of this on ESR's blog and was assured the science of how it works is accurate.
I looked up some of the info on this, and while I'm far from having a definite answer, I can say that it really depends on what wavelengths we're talking here. CO2 has a prominent absorption peak around 15 micrometres, which is already essentially opaque(10^-2000 transmittance and other silly numbers), and further increases in concentration will change absolutely nothing. At other frequencies, there looks to still be reasonable room for additional warming effects. I don't have actual numerical values here, though, just an estimate from looking at a couple badly-made graphs.
Here's Real Climate's take on the issue. Fairly interesting I think.
http://www.realclimate.org/index.php/archives/2009/11/the-cru-hack/
What's interesting? Most of it is spin that doesn't deal with specifics. The closest thing to a substantive point is where they claim that "trick" is a term for a technique, not an intent to deceive. But I don't think anyone read that comment any other way. The damning part was "to hide the decline." Then there is more spin about why the proxies don't work. Duh! We know the proxies don't work. That's the whole point.
It would be more interesting if the CRU emails didn't already demonstrate control of that website.
Linking to this garbage, propaganda site is like linking to socialist websites for unbiased economic coverage.
Here's some of there BS from the homepage:
"Observed global temperature changes remain entirely in accord with IPCC projections, i.e. an anthropogenic warming trend of about 0.2 ºC per decade with superimposed short-term natural variability." IGNORING that we've had over 10 years of no increase and some cooling. They use this "short term variability" to cover up the fact that their prediction were entirely WRONG for the period.
"The ice sheets are both losing mass (and hence contributing to sea level rise)." Ice sheets floating in the sea in the polar regions displace lots MORE volume than liquid water - just like ice cubes a drink. If they melt - the water level goes DOWN. I guess these hucksters missed grammar school science class.
"Perhaps most importantly, the report articulates a much clearer picture of what has to happen if the world wants to keep future warming within the reasonable threshold (2°C) that the European Union and the G8 nations have already agreed to in principle." Yeah! Follow us into the wonderful world of a socialist controlled economy, and ruin industry, in the name of "saving the earth".
These people are outright frauds, hucksters, liars, and lackeys for the socialists of the world.
Ice sheets floating in the sea in the polar regions displace lots MORE volume than liquid water - just like ice cubes a drink. If they melt - the water level goes DOWN.
That's not correct. Floating ice, like all floating objects, displaces its own mass in water. That's why ice sticks up above the level of the water. When ice melts, the level of water (ocean or in a glass) stays the same.
The issue with melting ice is the ice found sitting on land; it could indeed raise water levels.
The vast majority of floating ice is below and is highly expanded - increased dispalcement. Just do the experiment. Put a lot of ice cubes in and glass and fill it with water to the brim. Then wait for the ice to melt. Don't confuse sheet ice with icebergs.
The only "ice sheet" that I've ever heard of is the one in Greenland. All others referred to sea ice in Polar regions, especially Antartica. Greenland is a giant bowl. Much of any melt would be contained in the bowl - total melt creating a lake. Of course, since Greenland was warmer in the middle ages, supporting Viking communities, and since there is no record of any catastrophic flooding in Europe or anywhere else during this period, the huckters argument "to avert catastrophic flooding" does not hold.
These clowns will say anything to spread fear and panic.
The only "ice sheet" that I've ever heard of is the one in Greenland.
Antarctica. Big continent south of Australia. You may have heard of it.
ed, the simple fact is that anything floating displaces its mass in water. If it was displacing more than its mass in water, upward pressure would drive it out of the water until it wasn't anymore. If you're seeing a water level drop in your experiment, you're either seeing evaporation or you've crammed the ice in so tight it can't float upward.
Also, Antarctica is a land mass. There is sea ice there, but also a heck of a lot of non-sea ice.
Depends if the salt content is the same.
Fine, we'll add in salt content. Seawater is denser than freshwater, so a freshwater iceberg displaces LESS seawater, volume-wise, than it will be when it melts. So the melting of a freshwater ice cube in sea water will cause the level to rise, unlike the case where the salt contents match.
Well, no, about the ice sheets. You may have observed that when you have an ice cube in your drink, part of it stick up above the liquid surface? That's because the volume of water it displaces is the volume that has the same mass as the ice cube; since the ice cube is less dense, it displaces a volume of water smaller than its own volume. If it melts and turns into water, it will take up a smaller volume . . . precisely the volume of water it displaces. So the water level won't change. Try the experiment yourself with a measuring cub, some water, and a couple of ice cubes.
The change in sea level is due to the melting of glaciers that cover the land . . . for example, Greenland and Antarctica. Since those aren't supported by buoyancy, they aren't currently displacing any water. If they melt, the ocean level will rise. Such changes are documented, though mainly for ice ages when sea level falls.
Neither sort of ice's melting will cause sea level to fall. Making claims like that neatly encourages the belief of AGW supporters that everyone critical of their views is scientifically ignorant.
Salt water is denser than fresh. As far as I know, sea Ice, has greater expansion than fresh water ice and displaces more volume. I know of the normal 8-9% expansion ratio for fresh water, but not for salt.
The realclimate site that I copied/pasted said "ice sheets", not glaciers. Also, as stated above, Greenland topography is that of a bowl in the middle, so melt shold be somewaht contained in the bowl.
True, salt water is denser than fresh. However, I don't think that leads to the conclusion you want. If you float a less-dense object in a more-dense fluid, it displaces a volume of the fluid whose mass equals the object. If you melt that object into the fluid, the entire fluid gets less dense, and thus takes up more volume for the same weight. As such, ice(which does not contain salt) melting into seawater(which does) will actually cause a sea level rise, albeit a really trivial one because the difference in density between fresh and salt water is so low.
The only way melting would lead to a sea level reduction is if the floating object was made of a material more dense than the seas - sinking a steel ship, for example, would lead to a sea level reduction.
"Observed global temperature changes remain entirely in accord with IPCC projections, i.e. an anthropogenic warming trend of about 0.2 ºC per decade with superimposed short-term natural variability."
They're still clinging to GISS, eh?
Satellite says .1 degree per decade warming.
http://www.drroyspencer.com/latest-global-temperatures/
Dave you should read the opening page at Real Climate. He was summarizing the Copenhagen Report.
Brian:
The people of the CRU created realclimate.org.
The e-mails and code that was leaked/hacked shows them to be dishonest frauds.
Using realclimate.org as a cite to somehow refute that isn't a good idea.
Brian,
And Copehagen is based on...?
Whether it uses GISS temps or CRU, they have the same problems.
The satellites are much more trustworthy.
I wasn't refuting anything - just pointing out what the post was about that Ed originally linked to. Ed apparently has also never heard of the Antarctic ice sheet either.
This is interesting:
------------------------
From: Tom Wigley
To: Phil Jones
Subject: LAND vs OCEAN
Date: Fri, 06 Nov 2009 17:36:15 -0700
We probably need to say more about this. Land warming since 1980 has been twice the ocean warming — and skeptics might claim that this proves that urban warming is real and important.
----------------------
That argues strongly the satellite trend is the real trend, probably extending past the satellite record all the way back to 1850.
New rule: climate models must include source code and unit tests or be laughed out of publication.
Amen! Not just climate models. I find it seriously negligent that major journals would publish whose findings are based on code and data that can't be publicly reviewed. It really destroys the ability to independently verify conclusions.
Without that it's not science, just fancy assertion.
"I find it seriously negligent that major journals would publish whose findings are based on code and data that can't be publicly reviewed."
The emails reveal that "major journals" have been taken over by the same crew perpetrating the fraud scheme. Other "major journals" editors have been cowed into silence by professional threats ... as revealed in the emails.
A vast left-wing conspiracy that has been revealed by the whistle-blower who released the faked data and model code. We owe that person a huge debt of gratitude - much as we owed a debt of gratitude to the people who released the Pentagon Papers.
It's an epic story.
A vast left wing conspiracy? You have got to be kidding right? Those emails don't reveal any such thing. It's more reflective of what's going on in your head.
Well, maye not vast, but there was definitely a left-wingish conspiracy to keep skeptics out of journals.
Poor Brian,
Directed by his handlers to defend the scam. Yes Brian is is a vast conspiracy and the emails show it. It involves years of propagandizing, huge amounts of money (made by hucksters and cons like Al Gore), and very powerful government officials (Hansen at NASA!?!?). It encrapsulates an entire branch of science. ((What other scientists do we need to start doubting?? How about epidemiologists in these days of government health care?)) The scope of the theft was trillions of dollars and our children's futures.
I second my gratitude to that whistle blower who risked so much to bring the conspiracy tumbling down. Hope they make a Woodward and Berstein or Erin Brokovitch like movie out of it.
If American Republicans had done this you would be calling for the gas chambers.
Ed just because call someone a socialist doesn't mean they aren't right. There is always going to be some variability in rise because there was variability in rise in the past.
You are selectively quoting Real Climate - the report was referring to Greenland and Anarctica ice sheets which would lead to raised sea levels. Nice failure to read.
I've long been skeptical of the more catastrophic scenarios, because all this carbon used to be in the atmosphere, which probably defines a ceiling on how bad it will get
As much as I'm a skeptic who hates the doom scenarios, the ceiling's higher than that.
1) The current best estimates of the astronomers are that the Sun has relatively steadily increased its output.
2) It's at least possible that a reasonably high percentage of the carbon in the Earth's crust has been there since the beginning, having never outgassed into the atmosphere as the Earth formed. Most models of the Earth's early formation make this unlikely, but abiogenic coal can't be completely ruled out. If it exists, we could theoretically manage to add more carbon to the atmosphere than was ever previously present.
OTOH, we know CO2 levels have been more than 10 times higher than now, so it would take us a really long time to get there. I think we run out of coal and oil well before that happens.
The sun is rather quiet right now. Solar Cycle 24 is very slow in developing and is now projected to be rather poor in sunspot count to Solar Cycle 23. The Maunder Minimum ( a period of very little sun spots) occurred at the same time as the coldest part of the Little Ice Age. I wonder (non-scientifically) if something similar is beginning to happen?
One issue I have always had with AGW is evidence of warming on other solar system bodies. Mars' polar caps have been shrinking for years. One of Jupiter's moons is also showing signs of warming. Even Pluto retained it's atmosphere longer than expected. The standard answer by AGW proponents was that we understood Earth's atmosphere and we didn't understand the other solar systems bodies' atmospheres. That seemed to me to be too much of a "don't bother me" answer. If other solar system bodies are warming wouldn't you think they might have a common source?
The real problem is not with science, it is with the human proclivity to fear death then base the entire concept of life on death.
That denial is well documented, like climate change science is, but it is unpopular to talk about it outside the lingo of denial.
Thus, like the Piltdown man fraud, it gives fear a place to hang its hat.
If only we knew how big we really are we would work together in harmony in place of fragmenting in the face of reality.
Megan,
I really don't understand the conclusions you draw in your post. You point out correctly that the models which are supposed to be used to calculate the cost of global warming (and even if it exists at all) are totally worthless, and then go on to say that we should do something about it anyhow.
But how do you know? If the models are worthless... than you have no data to go on. In fact, your first statement that global warming is still likely true has no founding. If the data doesn't support it, then all you have is a hypothesis that the CO2 we're putting in the air is causing enough of a warming... and no data to back it up.
If the data is bad, the conclusions are bad. It's not that global warming doesn't exist, or that it may not be a problem... its that we can now say we DON'T KNOW!
"If the data doesn't support it, then all you have is a hypothesis..."
No what you have in that case is religious faith.
http://www.merriam-webster.com/dictionary/faith
Anon Y Mouse warned us over a year ago on this very blog!
http://meganmcardle.theatlantic.com/archives/2008/07/boundary_cases.php
You must be on your carbon tax hobby horse. Please pay close attention to the following:
Anthropogenic global warming is a hoax!
Thank you for your attention.
I've got to read the comments more ...
What's wrong with a carbon tax? It really is the best solution. The only question left is whether it's a solution to a real problem or not.
Are you really taking shots at people based on the contents of emails about problems with a project? If you were to look at my emails, you might see something like this:
"The script was having problems with a decimal weight value as opposed to
a whole number. I've fixed the code so that it will clip off the
fractional portion of the number."
Now, out of context that might look like I'm throwing away useful data. In fact, someone put a float in a string field defined as containing an integer. In addition, our measuring devices that generate this data don't have precision to tenths of a pound. No useful data is lost, but I'm sure that if I worked for the CRU, that quote would be making the rounds right now.
Using comments is even worse. I've written dozens of comments in my career that indicate that the way the code is written is poor. This has never meant that the code didn't generate the correct result (I would have fixed it if it did), but rather that it generated it in a way that was difficult to follow or suboptimal. If someone can point to genuine unfixed bugs that have a material effect on the output of the result then that's really bad. Until then, it's a non-issue.
. If someone can point to genuine unfixed bugs that have a material effect on the output of the result then that's really bad. Until then, it's a non-issue.
That would require the source code, which afaik, isn't available. You're right that using comments as a proxy for code quality isn't likely to be very accurate, but the ultimate proof of whether or not any of this is a big deal is the code itself (and the data). Without that it's impossible to know.
Even really good, experienced software developers make mistakes. That's why most software that gets shipped has a battery of tests that it needs to pass before release. Frequently more man-months go into testing and QA than development.
Most academic-type programs don't get anything like this level of scrutiny. In general that's fine, but once we start using these programs to make multi-trillion dollar decisions, the code should be open-sourced, and tested within an inch of it's life. It's well worth it to get the best information possible.
Pure obfuscation. There were entire datasets lost which were subject to a legal FOIA inquiry.
The comments I'm reading--admittedly, from developers who know more about this sort of thing than I do--basically seem to say that the debugger gave up trying to reproduce the results, because he couldn't; there had been too many undocumented manual interventions.
Every shop I've been in had some sort of source control, even if it was just zip files of the source tree, but usually something like CVS or Subversion. In applications where data was important that's been in source control too. Especially as data format versions often drive program versions. But these guys didn't use any source control. And now it's bitten them in the ass. Well, they're not the first coders to learn something The Hard Way. But they're getting it really hard. No lube, either.
It also looks like very noisy data (darned analog world! I can relate...) And very incomplete data, in terms of all the fields in the database being filed (I've run into that, too). Inconsistent location data as well. In general, very bad data. Has to be hand massaged to fix it up for the missing fields, bad locations, improper types, etc. And there's too much to hand massage, so they automate that, which means that the wrong fixes get applied to particular data, which means that the fixed dataset has lost it's connection to the real world.
"...which means that the fixed (sic) dataset has lost it's connection to the real world."
It is, in fact, no longer a "dataset"; it is an "un-dataset".
Did you intend "fixed" to mean repaired, or neutered? :-)
Repaired. I've had to write quite a bit of code to deal with real world analog data all of which had to be fixed to get it to work. Often in realtime.
had some sort of source control
This often isn't the case in areas of academia that aren't computer science. Especially at the time when this code was originally being developed.
I don't find that surprising either. There are a few import processes that I've written that required a bunch of manual intervention. I didn't automate every step because I was going to run it one, maybe two times, I knew what interventions to make because I was writing the code at the time. Certainly if I'd had more time to write the imports I would have done the "right way" but having been a scientific software engineer, I understand the really limited resources they are working with. Now, you could argue that "really important science" means that they should always do the right thing, but it's a matter of incentives. If the data ends up passing whatever tests they have in place, and it's much faster for me to do part of the manipulation by hand (say running a regex in vi to swap the ordering of some fields or remove some garbage rows), and I have 15 other things I have to do today, I'm gonna do it the wrong way just so I can move on to my next project. I will get yelled at for not getting job 15 done, I almost certainly won't get yelled at for not automating the entire load process.
And don't get me started about documentation in that sort of environment. The fact there is some at all is amazing.
Also, do note that while there may be serious bugs in whatever software system that's under discussion here, the odds of several independently written models all concurring is very low, absent some conspiracy. I've seen no evidence of one in all those emails, so it tells me that while they may have written crappy code that I would be ashamed to put my name on, it probably generates roughly the right results.
This isn't the model--it's the largest temperature dataset that underlies the model. GIGO.
To Megan:
It's the software that generates/massages the largest temperature dataset that underlies the model, no? Assuming a priori that because there are bugs the resulting output is unusable is fairly funny to me as a developer. I've discovered more than a handful of bugs that have ended up having no effect on the user due to other code downstream.
The point still stands re:other models concurring. What happens if that dataset is dropped from this model? Every single graph I've seen shows all the datasets showing the same basic trajectory, upwards. Until someone comes up with the bulk of the models failing to show warming when excluding this dataset, it seems a bit crazy to assume that the conclusions of their research is in danger. There have proxies from dozens of sources, from tree rings to ice cores all showing the same trends.
I too assume warming is happening. But their estimates are the largest set, particularly the older stuff, or so I understand it. Which calls a lot of the trend analysis into question. So while I believe warming is happening, I'm not sure we understand the size of the effect as well as we thought.
If I'm wrong about any of this, I'm happy to be corrected. But the general scientific rule is that you have to throw out results you can't reproduce, which they apparently can't--that's what the "rely on published" section seems to say.
You are almost right about the general scientific rule. If other people doing similar work can't reproduce your results then you have a problem. Reproduction doesn't mean doing the exact same experiment and generating the same results, though that is one (uncommon) way of doing it. What usually happens is someone else tries to come up with another experiment that will be able to confirm or deny your hypothesis so to as avoid duplicating work that's already been accomplished. So to stay with AGW, someone has some tree rings that show that the earth is warming. You could reproduce those results by taking tree rings from the same trees, or from different trees, or by taking the temperature data from ice cores. If both experiments say that the temperature 50,000 years ago X, then they have reproduced your results. Obviously it's a much stronger argument when you can say "I took these tree rings and ice core samples that say X" as opposed to "I have 3 sets of tree rings from the same trees and they all say X."
You're describing physical science like chemistry. In the case of social science, or anything else that uses data sets that aren't reproducible, the standard is data release.
Can you please explain why you deleted my comments? They were on topic, not insulting or otherwise untoward, and not in any violation of any posting standard I can see. They were also among the first posted and were up for quite some time.
Thanks.
My mistake, it's apparently a similar/identical story posted elsewhere in the Atlantic: http://business.theatlantic.com/mt-42/mt-tb.cgi/18209/
"The script was having problems with a decimal weight value as opposed to
a whole number. I've fixed the code so that it will clip off the
fractional portion of the number."
That's not analogous at all. That looks like a kludgy workaround. Theirs look like intentional attempts to hide or massage the data.
The email excerpt was about an issue where I had to massage the data, but in an uncomplicated fashion. The data I'm getting from one system is crap, so I'm manipulating it to ensure that the data I'm getting is usable. Their transformation could be a valid way of massaging the data they are using. Just because the data is massaged doesn't mean it's automatically invalid or no longer usable. It happens all the time when you have messy inputs.
"The data I'm getting from one system is crap, so I'm manipulating it to ensure that the data I'm getting is usable."
Usable CRAP, that is.
If you feed usable crap into a model, can the output be picked up by its clean end?
You shouldn't take crap in a literal sense. When I say the data I'm getting is crap, I mean it won't work without adjusting it to fit the expected inputs of my program. If it were unusable, it wouldn't be crap, it would be useless.
Lets say that every 15th row of our dataset has some know anomaly (maybe they tack on some logging information, or maybe the sensor resets periodically). Doing some sort of transformation on every 15th row to either correct or remove the bad data takes a crappy dataset and returns a clean one. Sure, we may be losing some information, but provided that the information lost is either irrelevant to our needs (say logging) or caused by a known problem with the source data (bad sensor that resets) we still have good data at the end. I don't think you'd argue that your local weather station reported useless data if every night at midnight they told you it was 100 degrees out if it was accurate the rest of the time. You'd just ignore what they said at midnight.
Of course, if they "accidentally lose" that data and never tell us they were correcting for it, it certainly places their "trick" into question, doesn't it?
I don't think this accurately describes what the comments suggest was going on. Yes, I've worked with bad datasets and legacy systems before, and I too have massaged them rather than take the time to fix the underlying problem. But we sure didn't do that with our market data servers.
Megan,
I didn't see anything from the examples in your post that made me thing that anything untoward was being done. Now, the programmers note is a bit more concerning out of context, but reading some selections of the README put it in a different light.
The net appears to be that the data has been massaged by hand and several different pieces of code over a long period of time and nobody remembers exactly how it all happened, so someone in CRU assigned poor Harry to go figure it back out. He spent a lot of time writing about being justifiably angry about how terrible that project is and a lot of time writing about solving problems. The file ends before he completed a description of how to regenerate the data which could either be he figured it out and didn't write about it or he gave up. No way of knowing with the evidence in that file.
So, I'm willing to call CRU out on not documenting their process (though I understand why they didn't) and I'm willing to grant that reproducing the process of transformations the data went through is probably lost (or at least the historical path the data went through is). The implications are these data, like all other scientific data, should be taken with a grain of salt and verified in other independent ways to ensure that there are no errors occurring. If this was the only dataset, I'd be very wary, but there are plenty of other independent datasets to cross check against.
I notice our usual left-wing trolls aren't posting on this thread. They must be feeling very scared about now that the AGW gig is up.
Oh, please. Do you really think that this gig is up in any meaningful sense? This will keep going for many more years, scandal or no scandal.
The global climatologist models are clear: the hilarity of climate predictions will continue increasing as time goes on, unless drastic action is taken. By 2100 the world may be laughing so hard the consequances for life on this planet will be devastating.
You should also notice that the major media are almost uniformly refusing to cover the story of this falsified data (CBSNews being a notable exception).
That's telling. It's a tacit admission that telling people about these emails ... allowing a discussion to begin on the underlying computer model code ... revealing the existence of the altered and faked data ... would totally undermine their past editorial efforts.
So, major media outlets have banded together and agreed amongst themselves to simply ignore this story. ("Climate journalism" is a fairly limited subset of people who all sort of hang out together anyway ... like the high school glee clubbers).
What is telling is the number of readers in the comments section who are crashing their little gatekeeping party. Readers know more about this issued than either reporters or editors and it is painfully obvious to everyone what is happening.
Gates. Crashed.
wins32767, I don't buy it. I think you must like them. The level of un-documentation (not Harry's, his predecessor's) shown in the harryreadme files would get me fired. You can play these games for a quick-and-dirty bug fix. Not for a major project. We all know what happens to projects that aren't properly documented or without software control: Just what happened to this one. If they can't reconstruct how they got their results, leaving aside question of negligence, the impact is that their results no longer have any scientific validity.
Now the rest of climate scientists have the job of decoupling all that work from everything else they've done, to see what's left. Plus they have to convince the rest of us that their own work is okay, by a whole lot more transparency than these guys showed.
It's not that I like them, it's that I've been in their shoes. The scientific world is very, very different from the business world. The number of projects that have actual software engineers on them is vanishingly small. Most of the code is written by scientists with no formal training in software development, which is why they don't use version control. Still, they get the right result on a regular basis, despite the contortions they go through to get there. Go take a look at matplotlib for an example of how things are done in the scientific world. That's a well regarded project but as a software engineer, it horrifies me. That doesn't mean it doesn't make accurate graphs though.
Just because they got a good result with one project doesn't mean that they aren't fudging the AGW stuff. There is literally trillions of dollars at stake with all this.
Let's be realistic here. Actual science is messy, much like law making. There is a reason every single scientist says "these data indicate". If you want absolutes, science won't be able to provide them. CRU is a messier than the grant I was on, but we were very well funded. The strength of science isn't with getting the results exactly right on every experiment, it's with doing lots and lots of experiments the best you can and seeing where they all agree. There are lots and lots of experiments that indicate we are causing climate change.
---Actual science is messy, much like law making. There is a reason every single scientist says "these data indicate". If you want absolutes, science won't be able to provide them.---
So with this lack of absolutes the entire western world - not China (building a coal plant a week) was supposed to change its ABSOLUTE standard of living....
http://www.nationalcenter.org/PRGoreEnvironment104.html
I never heard Nobel prize winner Gore say 'these data indicate'. He said "The Science Is Settled" and "Those who disagree are Nazi's".
http://www.timesonline.co.uk/tol/news/environment/article6658672.ece
Those statements are absolutes, coming from a Nobel prize winning climatologist and prophet.
Beg pardon, but it isn't true that "every single scientist says 'these data indicate'". There is a sizable percentage with doubts about various aspects of the consensus. As I mentioned earlier, John Christy is a good example, though there are many others. If I recall correctly, there are a lot more doubters among the data-gatherers like Christy than among the modelers, as one might well expect.
A problem here is that the vast majority of scientists in the field are not doing 'lots and lots of experiments'. There are a limited number of experiments that are providing much of the global data, and a lot of it was in this here database. Then there are a lot of scientists who are doing experiments or analysis, but not directly toward checking the basic hypotheses of the theory. Rather they're working on some detail or other.
It remains to be seen how the loss of this dataset will affect the whole field. And the dataset is lost. The fact that you can find ways to excuse their misconduct doesn't change the fact that no one can use results that can't be replicated. And even given full access to the raw data will probably not help, since they don't seem to have the original data any more.
"... since they don't seem to have the original data any more."
Of course, we're made to take their word for it that they don't have the original data any more. Or do they?
Not one fraudster in the East Anglia CRU has been put under oath to answer that question yet, and so we don't really know for sure whether the data is lost or more likely whether the data has been deliberately hidden to prevent the fraud from coming to light.
Tracks are being covered. Warrants need to be issued. Arrests made. Bottoms gotten to. The Queen's grants refunded. Gael's filled.
From an email sent by Phil Jones:
PS I'm getting hassled by a couple of people to release the CRU station temperature data.
Don't any of you three tell anybody that the UK has a Freedom of Information Act !
Megan,
We are discussing one of the key components of a process which would lead to a revolutionary change in the entire US/global energy economy. The US has literally spent $ billions studying this issue. The US would be required to spend $ trillions to reduce carbon emissions sufficiently to halt the increase in atmospheric carbon concentrations. To my mind, that makes this issue far more significant than "market data server" issues.
The fact that this situation is this bad and has persisted this long seems almost incredible. The suggestion that we keep moving forward with the COP15 process regardless IS incredible.
There is little question that much of the data collected to document global temperatures is fraught with bias error. We have recently heard the "dog ate my homework" excuse regarding the availability of the raw data from CRU. Both CRU and GISS have been resisting FOIA requests actively and passively for years.
It is long past time to pull back the curtain and analyze the "Great Oz" in all of his splendor.
I hope it is ultimately determined that the the "CRUtape" release was the result of action by a whistleblower; and, that the whistleblower is awarded a knighthood as a result.
wins32767,
You keep commenting to the effect that many other researchers have independently derived results very close to those the CRU group produced. I'm not sure that's correct. First of all, how many others have focused on paleoclimatology, as opposed to current trends? Second, how many studies and/or papers have been truly independent of the CRU conclusions? These are sincere questions, and the answers tell us a great deal about how much this all matters.
I understand your sympathy for the coders at work here, but that's irrelevant. I haven't seen much condemnation of the coders per se, rather of using what is apparently a broken mess as the basis of very serious scientific claims with vast real-world implications. If you want to change global policy, you should need to provide rock-solid evidence. Right now, the CRU group's evidence looks pretty shaky.
"Right now, the CRU group's evidence looks pretty shaky."
And were the issue only their faked data it wouldn't be a compelling story. Disreputable scientists fake experiment results all the time for reasons of greed and ego.
On top of their shaky evidence however, we have deliberate destruction of the supporting temperature data; illegal conspiracies to subvert Freedom of Information laws in both the United States and the UK; threats of retaliation against scientific journals that print alternate theories; and the creation of fake journals to suppress scientists who don't go along with the orthodoxy.
This story is, without a doubt, the biggest story in science since the publication of "De revolutionibus orbium coelestium." The cover-up is the most amazing assault on the scientific method since the Church found Galileo Galilei vehemently suspect of heresy.
Yawn. The problem with this thread is that the posters know a little bit about the issues. Enough to think they can state with certainty that AGW is a hoax, but not nearly enough to really evaluate the data.
Can anyone here actually list 100 sources of data that support the AGW hypothesis? Because there are thousands. CRU is one temperature record that demonstrates the issue most comprehensively, in one graph, for a global scale. There are plenty of others.
It's like the discussion of the MWP we had the other night. A lot of people are convinced that the MWP "disproves" AGW or that the MWP is unexplained by scientists. Neither are true. IT has been the subject of extensive research.
And it continues to be. I linked to a paper from September that demonstrated the regionality of the MWP in a way that seemed persuasive to me, a non-expert. This is the other thing that rankles so many of the non-ranters. Science is not static.
Do you have any idea what quantum mechanics looked like 20 years after Einstein's revolultionary 1905 paper? It wasn't pretty. We didn't even get Heisenberg Uncertainty Principle until 28,I believe. That doesn't mean it was ever wrong -- it wasn't. It was just not fully complete.
AFAIK, the only thing both the MWP and the LIA disprove is the "hockey stick".
Perhaps, someday, that may be the subject of a book entitled: "A Tree Grows in Yamal"; or not.
There was also once a strong consensus for the indivisible atom and Steady State theory. They turned out to be completely wrong.
What we didn't have in most earlier cases was a political agenda driving and distorting the science.
A lot of people are convinced that the MWP "disproves" AGW or that the MWP is unexplained by scientists.
It's highly inconvenient and poorly explained. Mann actually inverted a dataset to try to kill it, while Briffa used horrible data in Yamal.
Can anyone here actually list 100 sources of data that support the AGW hypothesis? Because there are thousands
Unforunately, a large portion seem to come from the same small group of environmental activists.
I wouldn't call it a "hoax" any more than Y2K was a "hoax", nor does virtually anyone I read. They think the risks are overstated and/or the alarmists' prescribed solutions don't make sense. But alarmists prefer to pretend those who disagree with them think it's a "hoax" or that a "grand" conspiracy is running things, whatever than means.
Alarmists use shocking tactics which should alert everyone they are not performing the role scientists are epected to perform. Among other tactics they are hiding results, using backdoor political pressure to force out competing views, and purposefully allowing their research to be misused. Let's all remember and laugh about Democrats who claimed they want to restore science to its rightful place, which is apparently all the science that fits the politcal narrative.
Looks like even Mann is conceding a global MWP now, with some local caveats.
http://wattsupwiththat.com/2009/11/26/mann-has-a-new-paper-he-apparently-discovers-the-medieval-warm-period/
By the way, I'm still waiting for the non-AGW explanation of the FACT of global warming. Nobody denies that the earth if warming. You say it's not carbon emissions. Fine. What is it? Solar radiation was disproved long ago and was never more than a crackpot theory.
What's the explanation for the MWP?
Heck, even the Younger Dryas doesn't have a clear explanation.
"By the way, I'm still waiting for the non-AGW explanation of the FACT of global warming. " I'm not a climate scientist, and I don't think anyone else here is either. As such, our opinions on the subject are _irrelevant_.
"Solar radiation was disproved long ago" - our opinions on the subject are _irrelevant_. I mean it. Please don't waste my time, and I don't want to waste yours. You have no idea what was proved or disproved, just what someone you trust said. Maybe I don't trust him.
The climate scientists have to finish sorting their field out, and they've got a lot of work to re-do after this last week. When they work it out, they will tell the rest of us, hopefully in a way we can rely on. Till then, your opinion and mine are, once again, irrelevant.
To some extent this is true. But (1) countries around the world are preparing to act NOW on alleged climate-change issues, (2) those actions will be dictated by politicians, not scientists, (3) we as citizens must weigh in on the political decisions, to the extent feasible, and therefore (4) we must come to at least tentative conclusions now, not ten years from now. Even if the current conclusion is "wait ten years, then re-evaluate".
And, of course, (5) we've been told for years, most prominently by Al Gore but also by prominent scientists, that the field WAS sorted out, here were the conclusions, shut up.
So I sympathize with your view, but can't agree.
Well, I agree with everything you said. We are forced to make decisions on insufficient information. I just try to recognize my limitations. And (at this points) humanity's. We would do better from a Bayesian point of view to put our money now into getting things straight, rather than trying to fix things we don't understand.
"Nobody denies that the earth if warming."
Many people do. The science is not settled and never has been.
For example, there is no universally agreed upon method of planetary temperature measurement for years where data was not collected. Scientists are unable to agree on a way to establish what the global temperature was in any given year. They can't do so even today.
Lots of competing proxy methods exist ... but all of them produce different results mostly tainted by the predisposed opinion of the advocate.
Well Al G.,
Thanks for your very scientific statements. They are up to your usual demagogic and sophistic standards.
Of course there are scientists that deny the earth is warming and those that say that warming is not anthropogenic. It is just old technique to flatly deny that your opponents even exist.
Nor has solar radiation been disproved, and just once I'd like a global warming religionist to tell me why Mars is warming as well as Venus.
--------------------------
The planet Mars is undergoing significant global warming, new data from the National Aeronautics and Space Administration (NASA) show, lending support to many climatologists' claims that the Earth's modest warming during the past century is due primarily to a recent upsurge in solar energy.
http://www.jpl.nasa.gov/news/news.cfm?release=2005-152
Pluto is undergoing global warming, as evidenced by a three-fold increase in the planet's atmospheric pressure during the past 14 years, a team of astronomers from Massachusetts Institute of Technology (MIT), Williams College, the University of Hawaii, Lowell Observatory and Cornell University announced in a press conference today at the annual meeting of the American Astronomical Society's (AAS) Division for Planetary Sciences in Birmingham, Ala.
http://www.spacedaily.com/reports/Pluto_Is_Undergoing_Global_Warming.html
You are of a piece with these intellectual thugs who denied their colleagues the chance to publish competing viewpoints. We must be on our guard for you liars.
Erm..."The sun is responsible for the earth getting warmer" is far from crackpot. For one thing, it's literal fact. The bit about cosmic radiation seemed too obscure to me, but direct luminosity would seem a perfectly plausible explanation.
I mean, speaking personally, I think that carbon emissions are probably imparting a long-term warming trend onto the earth. My skepticism tends to rest more on the issue of why we should give a damn.
Solar radiation was disproved long ago and was never more than a crackpot theory.
So why is Mars warming up? Its icecaps are melting, too, you know.
… everything we know about carbon dioxide indicates that it has a greenhouse effect …
Sorry, Megan. You are as wrong about this as you were about AGW in general(I’m assuming you believed in AGW – am I wrong?). Carbon Dioxide many times higher than today or the levels that are anticipated due from anthropological sources in the future has been present in the atmosphere in the past and no greenhouse effect has occurred. It’s a pity from the standpoint of wanting and needing to blame mankind(that stain upon Mother Gia!) for catastrophe but mankind as far as we know is not a causative agent in climate.
In the Dark Ages the plague was visited upon men by God’s retribution for sin – or so they believed. Blaming Mankind for climate change is the modern equivalent. Classic Freudian projection writ large.
I don't think you'd argue that your local weather station reported useless data if every night at midnight they told you it was 100 degrees out if it was accurate the rest of the time. You'd just ignore what they said at midnight.
“… if it was accurate the rest of the time …” But how would one know that the rest of the local weather station’s data was accurate? What the local weather station(or a pro-AGW climate scientist) presents is accepted on faith. If there is a glaring and obvious recurring error shouldn’t one at least entertain the notion that the rest of the data may be inaccurate – perhaps more subtly inaccurate?
Look at the absorption spectrum of carbon dioxide. It has a greenhouse effect. It's essentially a chemical high-pass filter. Not the strongest or most efficient around, but its first-order effects are warming, and there is no argument of that point I've ever seen that's even slightly coherent. The questions are "How much warming?", "What are the feedback effects?", "Is it a bad thing?" and "Do we care?". But the physical properties of carbon dioxide are really not up for debate.
Alsadius:
Did some further research in order to dispute and realized that I was wrong on greenhouse effects in prehistory. My apologies to you and Megan. May Truth always win the day.
Let me tell you how the scumbag bureaucrats are using this global warming BS to screw people right now.
You see the bureaucrats want more money for the federal flood insurance program because they got hit bad with the Katrina disaster - created by other idiot bureaucrats and their ludicrous levee building, channel building and all sorts of other garbage in New Orleans - plus a few lesser hits.
How could they get more money? Ah! Say that "global warming" is going to make the seas rise and increase hurricanes. Since the sea is going to rise and be aggravated by hurricanes, they expand the areas they designate "flood prone". It doesn't matter that these areas have never flooded in history - with written records back to the 1600s.
Once an area gets that designation, you can't get a mortgage without federal flood insurance. The feds get a whole new slew of premiums at about $1800 per year. That's exactly what the feds have done with Long Island, NY. People who live in areas that have zero record of any flooding, ever, are being hit with this. If they can't afford it, tough shit. The feds want their money. the odds of ever having to pay claims are almost zero. I have no idea how many other places have been targeted for extortion by the bureaucrats, but I assume that Long Island is not alone.
This is just a preview of the crap the feds will hit taxpayers with when they can foist things like cap & trade and all other nonsense on us, all in the name of saving us from ourselves.
But saving us from ourselves is what governments do best!
Software development under a constrained budget is almost always terribly messy. Statistical analyses have been done, showing that the rate of programmer errors is very high(although probably not nearly the rate of pundit errors): ~1/50 lines.
That said messy and poorly documented code, no matter how far from ideal, does not mean the code is not useful for its purpose. It is a well known fact that no matter what operating system you use, it is just one layer of hacks built on top of another. Practically, code is inherently messy and ad hoc.
But ultimately, a code-base can be treated like a black box. If the inputs and outputs match for the test data you provide, it can be verified. Verified regardless of whether the internal calculations are unnecessarily complex or not. In the same way you trust your OS despite its internal flaws.
Moreover, If you drive in a car, or fly in a plane, you are implicitly trusting your life to computer models that is at least as messy, if not more so. If that scares you, then you'd best move to a log cabin and start mailing packages.
So how do climate models do at their black-box testing, anyways? Put the data up to 2007 in, and see what it spits out for 2008. Or, if you want some real fun, look back to 1998, and see what their predictions for the next decade looked like then.
The more I look into this the more I realize that the criticisms being leveled are nothing more than innuendo, with absolutely zero evidence.
At best, these are are superficial criticisms of coding style. The decisions made were clearly bad from a code maintenance/ development efficiency perspective, but none of the claims actually show a bug in the code.
Beyond that, people are looking at a stolen version of code which is admittedly not the current version. There isn't really any way of knowing whether any problems were fixed in the version that is actually used.
But to answer your question, my understanding is that independent models have good agreement and they do a fairly decent job with climate. Although unsurprisingly, the climate models don't do a very good job at weather. It seems that most critics of AGW models fail to appreciate this distinction.
I wasn't criticizing the coding - I'm an awful programmer, even if I was in the mood to read their code I'd have very little useful information to add. I was accepting your premise that coding style is irrelevant if the results are good, and asking how the results looked. You asked for a black-box test of their code, I asked what the results were. I'd wager that they bet high pretty much across the board, and their late-90s predictions will be hilarious in retrospect. I don't have any actual data to back up that supposition, but I'd be surprised if it's wrong.
Anyone have links to something of this nature?
If the inputs and outputs match for the test data you provide, it can be verified.
It can be verified for your test cases, but in my experience test data and test cases are a very limited subset of what happens in production.
My, what a spirited debate.
Let me weigh in with a few characterizations.
CO2 driven global warming is a THEORY.
The underlying facts are:
1. CO2 is a greenhouse gas (ie, it absorbs [limited frequencies] of heat energy from the surface that would otherwise make it out into space).
2. CO2 is a weak greenhouse gas (very limited energy frequencies absorbed), and not enough molecules in our atmosphere to make a tinkers'damn.
3. To actually affect temperatures, some other mechanism must be in place to store the captured heat -- like convection transfer to other molecules (think water vapor).
4. The AGW alarmists assume some threshold event that would occasion a large increase in average humidity to provide the heat resevoir needed to support their dire predictions. Not only is there no reasonable science behind this puted threshold effect, no one can model what would happen if humidity levels did increase (more clouds, perhaps?). It hasn't happened (ever), so how can anyone predict it would happen and what the aftermath would be (like it might be self-limiting).
Here's a thought experiment for you all. Why do nighttime temperatures decline more in desert environments than in the tropics? Ans. Because there's enough water vapor to hold onto the heat (and there are more likely to be clouds to prevent the radiation escaping the atmosphere).
5. The paleo data strongly imply that CO2 levels are a FOLLOWER of temperatures -- not a predecessor. This is likely because, as oceans warm they release more trapped gases into the air (including CO2 which is in solution).
6. The net effect of mankind's contribution to CO2 levels is miniscule compared to mother nature's.
7. And, as has been observed by earlier commenters in this thread, CO2 levels have been much higher in the distant past -- with no observed ill effects.
Rob Lyman
Thank you, Rob. I was wondering when I would finally have a good excuse to bring this up in Asymmetrical Information.
They don't need to be normally distributed. In fact, they don't even need to be identically distributed. Thanks to the Lyapunov's Central limit Theorem. Averages of samples will be normally distributed if the averaged random variables:
- are independent;
- have a finite variance;
- its higher-order central moments obey Lyapunov's condition.
Which happens with almost every real-world distributions (there are pathological cases like the Cauchy distribution, but these are mostly curiosities).
Pretty cool, huh? If you think this sounds too good to be true, I don't blame you. I had the same reaction the first time I saw this result.
As a side note, for reasons that I'm too lazy to write right now, I would be willing to bet a pretty large sum of money that errors in temperature measurements are normally distributed. But they don't have to be.
I presume that systematic errors would be classed as "not independent". There is some reason to suspect that the CRU data involved systematic errors -- not saying it did, just that it may have. But it's difficult to check, now that it's gone and all.
You're right, it's been so long since I thought about that, it didn't occur to me.
I'm not totally sure it helps with the problem, though. Assume no systematic bias, and measuring stations which have random and independent but repeatable errors (maybe the factory QC isn't all that it should be). If the measuring station in New York reads a couple of degrees high for random reasons, can we really count on that being compensated for in the global average because one in Barrow reads a couple of degrees low? Does that require any kind of correction for the fact that the Barrow temperature is consistently lower than NY?
I don't actually know. I presume (hope!) the climate guys do.
To put it another way, suppose you're calculating average height, but you know your tape measures have subtle differences in length. If a particular tape measure is always used to measure short people (as classed by a rough visual estimate), and another is always used to measure tall people, can you really assume the errors introduced by different tape measures averages out in the end?
Where's SoV to hector us about conservative ignorance when he would be most useful?
Usually, he's at the bottom of the thread, the land the Reply function has not yet visited.
I hate the %#$@^* reply function, so that particular quirk I don't mind.
I like it - it means I can follow comment threads, and I can just hit ctrl-F and look for my name to see who has replied to me. Saves me having to read the whole thread to see people's replies.
Let's break this into smaller parts.
1) In the tape measure procedure you describe, the measuring errors would indeed even out given a sufficiently large number of samples. Even if the 2 tapes had 2 different underlying distributions, so long as those distributions obeyed Lyapunov's condition. The average of the two samples would converge to an unbiased estimator of the true mean. Furthermore, the estimator would be normally distributed given a sufficiently large total sample, with variance inversely proportional to the number of total measurement. So, many measurements, or single measurements made by many rulers, is the key here.
2) If the measurements had any systematic bias (i.e., were not accurate - see this), that bias would indeed be reflected in the sample average. There's no way to correct for this without additional information about nature of the systematic bias.
3) Suppose you, Rob, had a theory that people were getting increasingly taller each year. Suppose further that your 2 rulers, in addition to being imprecise, had a systematic bias. They would add or subtract an unknown but fixed number if inches to every measurement you made. If this were the case, it would render the average value of the 2 rulers meaningless. But you would still be able to ascertain if people were getting taller or not. That's because, in this case, you would be interested, not in knowing the average height, but the changes in year-to-year trend.
That's more or less the same case as thermometers which have an unknown systematic bias - you may not trust the temperature reading a given thermometer gives you, but if the readings increase in a 20 year period, that will reflect an increase in temperature that is independent of the the thermometer putative systematic bias.
Thanks for the ruler reassurance. It's not obvious and I'm too lazy to try to work it out.
you may not trust the temperature reading a given thermometer gives you, but if the readings increase in a 20 year period, that will reflect an increase in temperature that is independent of the the thermometer putative systematic bias.
That's all well and good for the time period when we have good thermometers, but the systematic bias is a real issue when we start stitching modern thermometer readings into historical tree-ring data (or even historical thermometer data, if the bias is changing over time.)
Right, those transitions are problematic. You have to know specific stuff about tree rings and the thermometers that were used.
I have to confess I wasn't expecting that 2 different people in the comment section would have learned the fairly advanced Lyapunov's version of the plain old Central Limit Theorem, which has more stringent conditions (requires random variables that are not only independent, but identically distributed).
I'm impressed... geeks. Now bow to the great beard!
The last time I ran into it, I was actually fussing about data to be presented by an expert witness at trial. The other sides experts were trying to use non-normally distributed data on CLT grounds, and we were challenging that by trying to show they didn't satisfy the conditions. Oddly, in my academic days, we didn't really use it because our data really were normally distributed.
Some of the posters here must have a vested interest in AGW. If you read the emails and look at the doc files, you can tell is is all BS. Sorry, you can make every kind of excuse you want but first CO2 is not the problem. There has been many times in our past Earths atmosphere had much higer levels of CO2 than now. These guy doctored data, concealed data and prevented information from being viewed in contravention to the Freedom of Information Act. What bullshit the writer starts out with. CO2 is more conductive of light than other clear, invisible gases which are much more abundant in our air. If you want to reduce green house gas, liquidate liberals.
R.W.Hamming, "The purpose of computation is understanding, not numbers."
Just the opposite for these guys.
Global Warming: figure the Baysean odds.
What are the chances that the earth needs a climate warning at exactly, exactly! the time that computation becomes so cheap that everybody can do all the computing they want.
An astronomically rare coincidence.
A better explanation is that warnings lead to funding, and computers are a way to produce warnings, so when computers become available you get warnings.
The programs that warn get the money. The ones that don't, don't.
Carbon dioxide is plant food.
Welcome to the party Megan.
The raw data is everything. The only reason not to release it is because they are fudging the results. Climate guys aren't real scientists.
Computer models can't even accurately predict the past. None of them are sophisticated enough. And, as all the AGW nonsense is based on them, I say the "science" (complete with scare quotes) is not settled.
However, check out the Iris hypothesis. (Although if you go to wikipedia, be mindful of the "consensus" view - kinda shaky these days.) Also, look into negative feedbacks (eg. I punch - you duck). The earth has lots of these. But, not according to the econazis. Everything is doom, doom, doom!
Consensus is just groupthink for - you don't need to check our numbers - these aren't the droids you're looking for.
And, don't mind that Al Gore guy behind the curtain.
"Computer models can't even accurately predict the past."
And there you have the truth of it. The first, the VERY FIRST test to verify a forecasting model's accuracy is so simple as to be childish:
(Historical data minus yesterday's data) as input MUST produce yesterday's data, or else the model is inaccurate. It's the ultimate test of a model: can it predict history, before we set about using it to predict the future?
AFAIK, not ONE climatological model has been able to do just that.
Even worse, there has been plenty of evidence that the models produce a single result (i.e. "we're all gonna fry and sooooon!"), regardless of the data input.
And now we know that the data has been falsified, contradictory data suppressed, and programming has been terrible, both in its design and its execution. And for this we must spend trillions of dollars?
Yeah, they have major problems even modelling phenomena like the Indian monsoons.
Also, they tend to be built against the CRU data.
"everything we know about carbon dioxide indicates that it has a greenhouse effect, because it is more efficient at passing sunlight through to the earth, than at allowing that energy to reradiate back into space as heat"
Well not quite. Check out Lindzen and Choi Aug09 GRL. They find from measured data that the outgoing radiation (cooling) is not affected by CO2 concentrations. This is exactly counter to the result found in the all climate models. The models only produce meadow muffins (dried bullshit).
That smacks of being a statistical fluke to me, or some sort of systematic error in the data. I haven't read the paper, but I will say that its result simply doesn't make sense.
These comments make me curious: how many folks who frequent this blog write or have written software professionally?
Me.
I've imported data from ill-written datasets to other ill-written dataset. And you know? I documented it very, very, very careful. Especially since -- what happens when they have MORE data to import? Or you need, for whatever reason, to re-import the old data?
I do.
I do also. I think one important distinction regarding clean and bugless code is the following: are you writing code just to yourself or do you plan on other people using it in the future?
If you're writing it just to yourself, the code will almost certainly be more idiosyncratic, less legible and less well-documented, since your main concern is usually writing something that works and not waste too much time with it.
Most of the climate change crowd will, I think, fit in this category. They spend most of their time programming and tweaking simulations of dynamical systems.
Second that. But if this massive and important organization treated its main dataset as "writing it just for themselves", they deserve to lose credibility.
I don't write code. I have, however, evaluated and refined forecasting models, professionally.
/raises hand
Coding, bad GCM (climate model) programming is one problem among others.
Another is simply what is the raw-data against which GCMs are tested: where is it?
As of September, CRU already claims to have lost it! And Jones email from years earlier indicates he would rather destroy it than let one FOIA requester Steve McIntyre have it.
A third problem is what happened to all the FOIA requests that were repeatedly, almost comprehensively denied? It turns out the FOI officers at UAE were coopted by these viciously paranoid scientists.
http://wattsupwiththat.com/2009/11/24/the-people-vs-the-cru-freedom-of-information-my-okole…/
SMOKING GUN EVIDENCE OF FOIA FRAUD.
Cry me a river. They won't provide you the tools to run their stolen code?
That would be like if I loaded your car into a flatbed truck and then started whining about how you didn't keep your keys in the car.
Well, except that the ethic of science has been public disclosure of data and methods, ever since the founding of the Royal Society. And that ethic has served us well for centuries. So it's as if you complained about the keys not being left in the car *in a society where cars are public property that everyone has a right to use*. Such a regime wouldn't work well for physical objects, but it works very well for information; it's what enables science to be self-correcting.
It's not that the people making those requests have a special right to the data; it's that the researchers have a general obligation to make the data available to anyone. It's part of pursuing truth; and that you mock it indicates that you not only don't care about truth but have no understanding of how it is attained.
I think you have the scientific method confused with something completely different.
Scientists share their methods and results with other members of the scientific community who are acting in good faith. They don't have any obligation to help show random unqualified people who are openly hostile to them how to use their stolen data. Who does or doesn't meet this criterion is generally up to them.
Really, whether they are obligated to share their data with other scientists depends on the conditions of their funding.* Science is very competitive and generally you or I have no legitimate claim to a scientist's data any more than we have a claim to a company's trade secrets.
It might be nice if we had more mandatory freedom of information, but the world you want is not the world we live in.
*For example NASA and ESA mandate data sharing as a condition on receiving funding.
This is a common misconception among scientists who oddly frequently turn out to have serious methodological flaws to hide. Every time I hear of a scientist refusing to turn over his data set on the grounds that the critics are just going to search for flaws, my spidey sense goes off--all of the recent famous cases of academic fraud and misdemeanors involve exactly this behavior. By comparison, look at Levitt and Dubner, or even John effing Lott, who handed over their data to amateur critics like Steve Sailer, and got schooled over coding problems. That's how serious scientists act.
Indeed.
Remember Bellesiles? He claimed to have lost his data in a flood... which was news to the people in his unflooded building.
http://en.wikipedia.org/wiki/Michael_A._Bellesiles
zosima, I presume your high school biology textbook's description of the scientific method included something about full disclosure? It may be fine and fair to withold data and models while you are still investigating, but once you go public with your claims, you are obligated to make the data and methods available so they can be reproduced and tested by others. Witholding from those you consider not to be acting in "good faith" may sound noble to a superficial ear, but in practice, you have abandoned science and established the foundations of an Old Boys' Club, if not an outright religious cult.
Remember when the consensus of every intelligence agency in the world was that Saddam Hussein had WMDs? Well, we took a major action based on that flawed intelligence and everyone on the Left was rightfully angered about poor quality intelligence, use of single sources, "being lied to", and "the rush to war." (This seems very close to what we are seeing with AGW)
One of the intel reforms recommended to avoid this in the future was analysis units set up to be contrarian.
So, the government should set up a climate unit of contrarians to take a realistic look at AGW research before we spend trillions on a problem that we find out to be not very serious.
I am sure our progressives here will agree that this is sensible, after their experience regarding intel about Iraq.
+1.
Since the time of Socrates, those who have sought the truth have used adversarial means to test hypotheses. This is the basic root of how science should be done.
A problem in this case is that "people doing scientific research" and "people recommending specific policy options" are overly conjoined, causing the scientists to have too-vested an interest in the specific outcomes of their researches.
"What are the chances that the earth needs a climate warning at exactly, exactly! the time that computation becomes so cheap that everybody can do all the computing they want."
Another example is that doctors discover all kinds of stuff that could be scary for babies in the womb right when ultrasounds become common. So, my doctor told me, don't worry - most of these things are things that are normal, we just found out about them because we could not see them 20 years ago. (cavities in the brain, holes in the heart, etc.) Most clear up by the time the kid is born or within 6 months.
The hospital handed out defect lists for babies and we only had one - hole in the heart. Felt horrible until the meeting with all of the other parents in the ward. Every baby had 1,2,3+ problems.
Now, you could say "OMG, it must be the air quality (or whatever)" or you could say "Hmmm, ultrasounds, DNA testing, etc. simply means we know more about what would have been 'normal' 30 years ago."
I was born with a hole in my heart. Closed up on its own before I needed surgery.
I can just see someone quoting your "I was born with a hole in my heart" out of context next time they want to call you an evil libertarian...
"The Real Problem With the Climate Science Emails"
and comments
Is that they are too damn long, and repeat themselves. :)
The scientific computer computation issues can now be
"settled" by releasing the data sets and modeling code
into the Web Cloud and waiting for the open source results.
Consulting Stephen Wolfram would also be a good idea;
He could give you a true expert opinion as to whether
the climate is now, or ever can be, modeled well enough
to justify the sorts of corrective actions proposed.
My take: The whole issue will be rendered moot by economic
collapse before the AGW crowd can get the payoff of their
long con. Even short of collapse, their is no spoonful of
money to spare; Grandma needs SS and MC.
Megan,
I took a closer look at the actual code and the "VERY ARTIFICIAL" comment is completely out of context.
First, the comment is in IDL code not Fortran code. The originator of this rumor apparently couldn't tell the difference.
Second, it is in a personal user's code(last-name:briffa), not part of the climate model, at all. As far as I can tell without being familiar with the science, he is doing autocorrelation analyses on tree-ring data.
Third, it is an earlier one of six files whose names indicate a series of iterations of some code that was developed for creation of plots. This section doesn't exist in the later versions which are more clearly more carefully developed and commented. This suggests that the draft version with the comment wasn't used. You can also tell it is an earlier version because the original timestamps are still present.(mar 1999 for the version with the comment, jan 2000 for the final version)
Fourth, in the file in which the comment appears. The "fudge factor" variable that is created isn't ever actually used. It is created, and never actually touched again.
This is probably just some test code, the equivalent of scratch paper.
Whether you are aware or not, closer examination indicates you are participating in a highly unprofessional and misleading smear campaign. I know we disagree a lot, Megan. But this is way beneath you. You should publish a correction.
Third, it is an earlier one of six files whose names indicate a series of iterations of some code that was developed for creation of plots. This section doesn't exist in the later versions which are more clearly more carefully developed and commented. This suggests that the draft version with the comment wasn't used. You can also tell it is an earlier version because the original timestamps are still present.(mar 1999 for the version with the comment, jan 2000 for the final version)
Fourth, in the file in which the comment appears. The "fudge factor" variable that is created isn't ever actually used. It is created, and never actually touched again.
Whether you are aware or not, closer examination indicates you are participating in a highly unprofessional and misleading smear campaign.
This is mostly speculation. This can all be settled if CRU releases their code.
"This is probably just some test code, the equivalent of scratch paper": then they should release their other stuff and then we'll know.
Well going to the original data sources would be a reasonable idea, except that in a lot of cases the data has been "adjusted" when the results of real temperatures did not match model predictions. We now apparently use a lot of proxies instead of the original data sources and the public sources cannot be trusted any more, unless you know EXACTLY in what way they have been "adjusted". Some of the various ways the data have been adjusted, and a whole lot about how we obtained the false consensus can be seen at the following link.
BTW the author, as opposed to most of the those writing about AGW, is in fact an atmospheric science researcher.
hxxp://arxiv.org/ftp/arxiv/papers/0809/0809.3762.pdf
(replace hxxp with http to see the link)
I'll admit that I've written lots of science/engineering code, though as a researcher not as a professional programmer. I add that (1) My code was far better done than CRU's, and (2) Some has gone on to be commercialised, which was achieved by working with firms that had professional programming expertise.
I'll also add that I've had 40 years of experience of modelling interacting chemical and physical phenomena, a decade of experience with using CFD code, and more than a decade of experience with temperature measurement in the laboratory.
I also have had useful education and experience in relevant statistical methods, and a few experiences with scientific crooks.
This lot probably adds up to a background for understanding the Global Warming issue that's broader than that possessed by many "Climate Scientists".
Conclusion: the accumulated evidence so far is far too feeble to justify spending trillions of dollars. Not even close.
"Whether you are aware or not, closer examination indicates you are participating in a highly unprofessional and misleading smear campaign. I know we disagree a lot, Megan. But this is way beneath you. You should publish a correction."
What exactly do you claim is a smear?
" comment is completely out of context. "
Sounds familiar, where have I heard that? Real Climate? You might want to consult sources other then those controlled by CRU and the alarmist crowd for facts.
As fas as the smear comment, calling out those who hid and manipulated data, colluded to get rid of those that had differing opinions and in effect rid peer review of any but alarmists, by smearing them "deniers", isn't a smear, it's stating fact. It's beed documented elsewhere. See Lindzen as early as the first IPCC report. Megan stated rightly that this is very poor behavior, which needs to be addressed. The only smear here is what's been done by the AGW crowd for WAY too long to anyone who has a differeing opinion on the need to spend Trillions and remake a large part of our society. AGW alarmism is based on poor models, which dont predict the last 10 years correctly, much less whats going to happen 100 years from now. It needs to stop.
Whether or not AGW is plausible or not, it's an illusion that the science has been "proved" or that a consensus actually exists to anything like the degree claimed in regards to the alarmist view of AGW. Before I'm willing to funnel trillions of dollars to institutions of such great fiscal management as the UN, I need more then "the dog ate my data" excuse given by CRU. I need to see REAL peer review, not political maneuvering.
The current models dont accuarately predict results. Most scientists would at that point ask what needs to change about the model NOT what needs to be done to the data to make it fit the models. Many have pointed out above, that the code and email clearly indicates the solution at CRU, if you don't buy it, welcome to the realm of the skeptics, er deniers, er skeptics. There are many many other sources, IPCC included, that clearly show this is a "science" that has been involved in adjusting the data to fit the model for a long time. Until honest objective science, and not politics, is involved in the realm of AGW, I will treat it as I do all political commentary, where the objective is to separate my money from my wallet.
If you are in fact an honest skeptic. Congratulations. I would recommend keeping an open mind, and challenging you belief that the science performed to date has been openly and honestly carried out and that it does in fact say what you think it does, or what the IPCC summaries claim it does. A lot of people are going to need more then CRU's or GISS's word for it after this, or for that matter their proxies, like real climate. I'm one of them.
Zosima, I don't understand you. Complaints about one comment don't hold up under careful examination (good work, by the way), therefore everything written about the harry readme files is part of a "highly unprofessional and misleading smear campaign". Megan is suggesting that some IT professionals who've looked at those files conclude that the data and algorithms seems to be garbage. You've got a lot of work to do before you refute that.
In general, you seem to be proceeding from a point of view that the burden of proof is on the complainers. I don't see why. Like any expert witness, scientists have the job of convincing the non-expert jury. If we ask for more clarification, they'd better be willing to provide it, or they're out. Giving us speeches about how their opponents are (1) non-expert and (2) insincere may convince their friends. If it doesn't convince the rest of us, it's up to them to do a more open job.
Whether or not every scientist always provides his data, if someone asks for it and he refuses to provide it, he has no grounds for complaint if his work is not accepted. Why should it be?
In this case, replicating the work is not much of an option, or at least an extremely expensive one. They will have to release their data and algorithms or no one will listen to them in future. I don't see how that can be a bad thing.
The code is certainly horrifyingly bad. I've worked on code in an academic research lab before, and I'm aware that it's often not the best code in the world. Our code was never anywhere near this apallingly bad though- perhaps it helped that we were a lab in the CS department, staffed by CS grad students.
But there's a much bigger problem with not just this model, but all of the GCMs. There's absolutely no reason to think that it is possible for us to predict future climate with them. They might be useful for building intuition about some things, but their predictive power should be assumed to be nil. I first got interested in global warming about 9 years ago, and it was because I thought that some of the claims being made by modelers were hard to swallow, to say the least. The more I read about what they were claiming the more I became convinced that what they were claiming to be able to do was simply impossible. So to me there's little difference between a GCM with terrible crufty code and one with really pristine code.
Climate scientists are very jealous of their professional status, and very quick to claim that laymen could not possibly understand what they are doing, or critique it. But in fact they draw on a lot of other disciplines in computing, the physical sciences, and mathematics. And my experience has been that when people from these disciplines actually take the time to look hard at what's going on in climate science as it relates to their disciplines they are often horrified, even if their ideological commitments lean toward the WWF side of the spectrum to some degree. As one physicist said to me "These guys need some adult supervision."
I can't claim to be able to tell you what the results of a doubling of atmospheric Co2 levels will be (my hunch is that the sensitivity will turn out to be less than one degree Fahrenheit, but I can't give you any really convincing arguments for that.) But I can tell you that neither can the modelers. The climate is too complex and too poorly understood for anyone to be able to tell you that based on computer models.
If one good thing comes out of these emails being released it will be that people stop being afraid to cast a critical eye over what people in this field have been doing. Because their work simply cannot stand up to scrutiny. They've done a great job of making people afraid to look like knuckle-dragging young earth creationist types if they express doubt, but actually a lot of the "skeptics" are people who are expert in fields that are crucial for the case of people like Mann and Jones, and the whole Wigley crew.
I'm glad the thread is still alive and kicking. There's a very short video that sheds light on the particulars of the type of peer review process that's going on the climate change community.
I can't emphasize this enough: it really is mandatory viewing if you expect to have an informed opinion on the subject. Ignore it at your own peril!
Wow. CRU's research director has a serious attitude problem. OTOH, I'm definitely buying the next issue of Scientific American. Didn't know they published original research.
Confession: I've never been able to understand a single article in Scientific American.
They're written so as to make no actual sense.
That would be perfect for global warming.
Jeffery so cute!