WhyNot?

grading expert opinion

Category: Culture
Responses: 2 (2 in support, 0 neutral, 0 in opposition)
Number of views: 407
Tracking: Track this idea
Community Rating:Average AverageYour Rating:

a four (4) point screening test to weigh the soundness of proposed expert opinion:

1.Is the claimant credentialed & in good standing within the field that he is making the claim? (1 or 0)

2. Has claimant’s statement(s) been submitted to peer-review? And is the review accepted with a majority of reviewers within the field? (1 or 0)

3. If no to above, is claimant’s statement(s) within two standard deviations (2 SD rule) of a representative sample of the field? (1 or 0)(note: the 2SD rule comes from Ian Ayres book SUPERCRUNCHERS")

4. Does the claimant have a record of predictive claims? What is the track record (accuracy) of these predictions?

plug in the score of the 'expert' and receive a weighted (i.e. predictive accuracy is given more weight) ranking of the expert's claim.

ER = A+B+C+D x PR

ER = Expertise Ratio (higher is better)PR = Prediction ratio

Examples: 1+0+0 = 1 X 0.25 = 0.25 Very low ranking – Not worthy of review nor acceptance 1+0+1 = 2 x 0.45 = 0.9 moderate ranking 1+1+1 = 3 x 0.33 = 0.99

remember a high score does not necessarily entail objectivity, only that soundness of opinions are not all equal!

While this is very crude it may at least provide food for thought on how to filter the more sound from the less sound claims being made constantly. I originally worked on this after reviewing how my emotions differed greatly between the 'new's segments' & the weather report. i truly felt informed during the weather report, whereas i felt the range of emotions (anger, worry, sadness, fear, etc)during the rest of the news broadcast.......

spaine, Aug 19 2009

What do you think of this idea or comment?
(You can change your vote at any time)

agree I agree no opinion No opinion disagree I disagree

Users who liked this idea also liked:

Aerial pictures in flight (218 votes) Very strong
GPS and Digital Photos (173 votes) Very strong
Elevator DeSelect (161 votes) Very strong
Ending the war on drugs (247 votes) Very strong
Airport charity (148 votes) Very strong
This concept is great! (141 votes) Very strong
Good Samaritan Switch (59 votes) Very strong
Door locks (37 votes) Very strong
Who looks like ME (22 votes) Very strong
Free printing (19 votes) Very strong

Other ideas in category (Culture):

Comments from other members:

Add your comment

i forgot to mention that if this proposal is sound it could be used as a screening tool for all newscasters and pundits on our national airwaves. Just like in baseball we have measures of a players ability that have become more and more accurate over time (i.e. OPS 'on-base + slugging' is more telling of a players offensive skill than just batting average. or likecollege football's BSC ranking system. the networks using an independent ranking firm could assign a broadcaster's 'rank' by the ranking criteria. sort of a 'warning label' for information

spaine, Aug 20 2009

I'm not exactly sure what kind of expert opinion you're talking about? Is this person testifying in court? Making business decisions? An opinion is far better justified by a logical argument than by a mysterious score.

This fine formula looks like another example to me of trying to make judgement seem scientific by using math. An opinion is still an opinion. Financial guys charge millions for opinions, yet a baby with a dart board can pick stocks almost as well. And Oh yeah, the financial guys use nice looking formulas. I'm sorry I'm so critical--I guess I'm a cynic lately.

hrench, Aug 25 2009

I have a suspicion that Google already uses something like this for users as well as websites. It makes sense when you're trying to decipher the truth or legitimacy or intention of a hierarchy of information and its sources. I doubt Google would be willing to discuss this but I'm willing to bet they have a lot of experience with this.

painperdu, Aug 27 2009

I imagine the algorithm would be something similar to http://www.20q.net/

painperdu, Aug 27 2009