We’ve had some trouble recently with posts from aggregator links like Google Amp, MSN, and Yahoo.
We’re now requiring links go to the OG source, and not a conduit.
In an example like this, it can give the wrong attribution to the MBFC bot, and can give a more or less reliable rating than the original source, but it also makes it harder to run down duplicates.
So anything not linked to the original source, but is stuck on Google Amp, MSN, Yahoo, etc. will be removed.
How much are you paying for the MBFC API? The page says it isn’t free. I’ll give you an API endpoint which will check sources against https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources/Perennial_sources, if you pay me half of whatever you were paying MBFC previously. That list is quite a lot better than relying on MBFC.
I already scraped the list. It’ll take around an hour for my script to finish going down the sources and assigning web sites to each one, but I can have a working API endpoint for you tomorrow morning. I can do the bot part also, if you prefer. That’s probably easier than making a new endpoint and hooking it to a bot and debugging the connection and all.
Like I said, I think the idea that readers won’t be able to determine that Breitbart is unreliable is missing a pretty big elephant in the misinformational room. If the issue that’s causing you to keep MBFC is finding a better source that’s programmatic, though, then solving that is almost trivially easy and at least seems like some kind of step forward.
MBFC API is free as they gave us access for us as a Non Profit.
We already had in mind adding these sources to our bot but we didnt had the time and knowledge how to scrape that. Personally i would like to host it on our own server so that we dont require you to use your own money just for one bot, in what programming language did you write it?
Thanks a lot!
Rooki
Since it’s a MediaWiki page you can get Markdown source of the page with appending
action=rawquery to the URL.To be honest, that’s Rooki’s deal, but I’ll link them to this comment!
I’ll send them a link and an example of how to use it tomorrow.