Epistemic status: Speculative. I have no special knowledge about factcheckers, these are just my opinions from casual research of easily available public sources.
Here are some beliefs I have about factcheckers:
1. Factcheckers can’t do their job unless they are *exceptionally* vigilant against bias, sloppiness, and inaccuracy.
2. Factcheckers tend to be kinda transparent, but not exceptionally so, *and* they could do much better if they put some effort in.
3. There are probably ways to incentivize factcheckers to put some effort in, and do much better. I hope my lovely cadre of clever, thoughtful FB friends have some ideas.
Not really going to explain or justify 1 unless prompted, except to say that I’m much more worried about bias in the general culture of the factchecking organizations than about blatant ‘pay-for-play’ from funders or deliberately misleading reports.
As for 2.. take Politifact for example; it discloses that it is a project of the Tampa Bay Times, but there’s very limited information on the TB Times anywhere on their website. I could only find a small blurb that reads more like an advertisement for the paper. The TB Times (formerly St. Petersburg Times) is a reasonably liberal paper, and has openly endorsed Democratic Presidential candidates in every election going back decades.
I’m not saying that compromises the independence of Politifact, but it should be openly and publicly acknowledged! If they really want to be transparent, they should be *concerned* that such a connection could undermine their credibility and should make efforts to point it out and explain why it’s not an issue. Instead, this possible source of bias is sort of tucked away. That’s standard practice as journalism goes, but that’s *not good enough* for an organization claiming to be a uniquely objective ‘fact-checker’.
(As an aside, the Poynter Institute mentioned in Jake Krycia’s comment owns the Tampa Bay Times and derivatively, Politifact. It would be nice if the Poynter institute noted its ownership of one of the most prominent US factcheckers next to its pledge.)
Also, while Politifact gives a statement of its policies, there is no enforcement mechanism or systematic review of previous ratings to check for objectivity. I’m satisfied that they do a good job ensuring that the quality of their evaluations is reasonably good (sources cited, genuine attempt to find objective information), but, again, what’s good enough for journalists is not good enough for factcheckers. They should actively guard against non-representative selection of facts to scrutinize and sources to cite, as well as the subtle ideological influence of their staff’s personal beliefs.
For a contrasting, positive example, Factcheck.org give a highly detailed and thorough account of its funding (https://www.factcheck.org/our-funding/). Buttt, there’s no real explanation of its methodology or actual disclosure of the organization’s internal practices. If we were just worried about ‘paid schills’, disclosing funding would be good enough, but that’s not the real issue here.
3. is trickier and I’m a little reluctant to post my own thoughts for fear of influencing other commentators. Probably I’m just being way too optimistic about how seriously anyone takes my facebooks posts, and I should just chill and share. For now, I guess I”ll just say that GiveWell is an incredible organization and is the gold standard for transparency everywhere. It’s tempting to say “well, not everyone can be GiveWell”, but if anyone should be GiveWell, it’s factcheckers. GiveWell has done a good job of creating incentives to keep itself transparent, and factcheckers could learn a lot from their example.