Fascinating discussion. I found it very valuable as a former analyst (non-IC). There are lessons to be learned from this that are broader than the direct subject matter.
Very good. I was an army intelligence analyst as a young fellow (British). I always remember the Chief of Staff of a special forces regiment puffing his cheeks out and saying he was fed up of the intelligence people always telling him what was going to change, what was at risk. He wanted to know what was stable and not going to change because he could make plans around that.
I'd also say that forecasting, assessment and prediction is such at the heart of this that some idea ought to exist in the community of what makes a good forecaster. It's one thing to make sense of what's happened or happening (that's analysis) but another to predict where things are going. Tetlock's work shows that the big mistake to make is to assume that experts on, say, Syria, are the people best equipped to forecast what's going to happen in Syria. No. He found that the skills and character traits necessary for good forecasting are unrelated to subject matter expertise, which in fact may be a negative. I have worked over the years with zillions of experts on this, that or the other, but I really can't think at all of one who accurately forecast very much.
I would think if the community was abreast of Tetlock and someone like Nassim Taleb then some things might improve.
Santi, if you’ve never heard of the Expeditionary Force books, they have a wonderfully hilarious alien species where a near-universal society-wide gambling addiction serves as a key mechanism of information exchange — ESPECIALLY for high level intel — as well as a sort of evolutionary “remixing” algorithm that periodically shakes up entrenched power.
Also leads to some pretty epic gags, like their concept of a “gambling problem” meaning that someone has lost all enjoyment from gambling.
Fascinating discussion. I found it very valuable as a former analyst (non-IC). There are lessons to be learned from this that are broader than the direct subject matter.
Great interview!
Fascinating, thank you for sharing!
Very good. I was an army intelligence analyst as a young fellow (British). I always remember the Chief of Staff of a special forces regiment puffing his cheeks out and saying he was fed up of the intelligence people always telling him what was going to change, what was at risk. He wanted to know what was stable and not going to change because he could make plans around that.
I'd also say that forecasting, assessment and prediction is such at the heart of this that some idea ought to exist in the community of what makes a good forecaster. It's one thing to make sense of what's happened or happening (that's analysis) but another to predict where things are going. Tetlock's work shows that the big mistake to make is to assume that experts on, say, Syria, are the people best equipped to forecast what's going to happen in Syria. No. He found that the skills and character traits necessary for good forecasting are unrelated to subject matter expertise, which in fact may be a negative. I have worked over the years with zillions of experts on this, that or the other, but I really can't think at all of one who accurately forecast very much.
I would think if the community was abreast of Tetlock and someone like Nassim Taleb then some things might improve.
Santi, if you’ve never heard of the Expeditionary Force books, they have a wonderfully hilarious alien species where a near-universal society-wide gambling addiction serves as a key mechanism of information exchange — ESPECIALLY for high level intel — as well as a sort of evolutionary “remixing” algorithm that periodically shakes up entrenched power.
Also leads to some pretty epic gags, like their concept of a “gambling problem” meaning that someone has lost all enjoyment from gambling.
INR are legends for being the best analysts. Their org is flat. They avoid the ‘promotion to management’ problem.
The intelligence community lost all credibility in the mid-late 2010s when anon guys in their moms basement were conducting better analysis.
Prime example - 4chan vs Shia LeBeouf