Data sources
Every external source, how it is used, how much it counts, and whether it is working right now.
As of 12 Oct 15:54 UTC · methodology 1.0.0-rc2 · sources
News feeds, live status
| Feed | Tier | Prior | Status | Last error |
|---|---|---|---|---|
| CoinDesk | Crypto reporting outlet | 1.15 | Healthy | |
| CoinTelegraph | Crypto reporting outlet | 1.15 | Healthy | |
| Decrypt | Crypto reporting outlet | 1.15 | Healthy | |
| Bitcoin Magazine | Crypto reporting outlet | 1.15 | Healthy | |
| The Block | Crypto reporting outlet | 1.15 | Healthy | |
| Blockworks | Crypto reporting outlet | 1.15 | Unavailable | empty response |
| CryptoSlate | General crypto media | 0.95 | Unavailable | 403 Client Error: Forbidden for url: https://cryptoslate.com/feed/ |
| NewsBTC | General crypto media | 0.95 | Healthy | |
| AMBCrypto | General crypto media | 0.95 | Unavailable | 403 Client Error: Forbidden for url: https://ambcrypto.com/feed/ |
| CryptoNews | General crypto media | 0.95 | Healthy | |
| BeInCrypto | General crypto media | 0.95 | Healthy | |
| CryptoPotato | General crypto media | 0.95 | Healthy | |
| Bitcoinist | General crypto media | 0.95 | Healthy | |
| U.Today | General crypto media | 0.95 | Healthy | |
| CoinGape | General crypto media | 0.95 | Healthy | |
| DailyHodl | General crypto media | 0.95 | Healthy | |
| CoinJournal | General crypto media | 0.95 | Healthy | |
| Crypto Briefing | General crypto media | 0.95 | Healthy | |
| Coinpedia | General crypto media | 0.95 | Healthy | |
| ZyCrypto | General crypto media | 0.95 | Healthy | |
| GoogleNews: Crypto | Search aggregator (items re-attributed to real publisher) | per publisher | Healthy | |
| GoogleNews: Bitcoin | Search aggregator (items re-attributed to real publisher) | per publisher | Healthy | |
| GoogleNews: Ethereum | Search aggregator (items re-attributed to real publisher) | per publisher | Healthy | |
| GoogleNews: Regulation | Search aggregator (items re-attributed to real publisher) | per publisher | Healthy |
Reddit and other social sources are not collected in V1 (news-only scope). Adding them is a methodology change that requires review and a new version.
Source tiers and priors
| Tier | Prior |
|---|---|
| Primary / official | 1.3 |
| Major financial press | 1.3 |
| Crypto reporting outlet | 1.15 |
| General crypto media | 0.95 |
| Aggregator / repost | 0.8 |
| Social (Reddit) | 1.0 |
| Unknown publisher | 0.85 |
Priors are subjective methodology choices, not validated credibility scores. They are deliberately kept in a narrow 0.8 to 1.3 range.
Data health
| Crypto news feeds | Healthy | 22 of 24 feeds · newest item 0.1h old |
| Reddit (not part of V1, news-only scope) | Out of scope | 0 of 10 feeds |
| Macro headlines (context) | Healthy | 4 of 4 feeds · newest item 0.1h old |
| CoinGecko prices and trending | Healthy | |
| Fear & Greed (external) | Healthy | |
| Google Trends search interest | Unavailable | |
| Economic calendar | Not configured | |
| Funding and open interest (context) | Healthy | |
| Sentiment model | FULL MODEL ACTIVE | CryptoBERT + VADER ensemble |
Source inventory
V1 scope: news only. Version 1 measures crypto news coverage. Social media (Reddit) sentiment is not collected or published. Every reading comes from one data category, so nothing here is presented as confirmation across independent data types. Enabling social sentiment later changes the published reading and requires a methodology review and a new version.
Verified from code (config.py, collectors/, pipeline.py), not from earlier documentation. No sources were added in the v1 audit.
| Source | Provider | Type | Auth | Frequency | Fields used | Timestamp | Required | Failure behaviour | Main limitations |
|---|---|---|---|---|---|---|---|---|---|
| 20 crypto outlets (RSS) | CoinDesk, CoinTelegraph, Decrypt, Bitcoin Magazine, The Block, Blockworks, CryptoSlate, NewsBTC, AMBCrypto, CryptoNews, BeInCrypto, CryptoPotato, Bitcoinist, U.Today, CoinGape, The Daily Hodl, CoinJournal, Crypto Briefing, Coinpedia, ZyCrypto | News | None | Every 10 min, 15 s timeout per feed | title, summary, link, published | Feed published, else updated. Missing = flagged estimated | Core | Feed listed as failed. Coverage drops to Partial | Feed depth varies (10 to 50 items). Several outlets publish heavy price commentary and sponsored posts. |
| Google News search (4 queries) | Aggregator | None | Every 10 min | title, <source> publisher, link, published | Feed published | Optional | As above | Max 100 results per query. Re-attributed to the real publisher, many of which are reposts. Links are Google redirect URLs, so URL dedupe across outlets relies on headlines. | |
| Macro search (4 queries) | Google News | Macro context | None | Every 10 min | as above | as above | Optional | As above | Context only. Never in crypto tone. |
| NewsData.io | NewsData | News API | API key(s) | Budget-spread, about 180 calls per key per day | title, content, link, pubDate, source_id | pubDate (assumed UTC) | Optional | Skipped without a key. Rate limits rotate keys | Free tier delays. Publisher tier often unknown. |
| Reddit, 10 subreddits (not collected in V1) | Social | PRAW app credentials optional | Every 10 min, 100 hot posts each | title, selftext, score, comments, link URL, created time | created_utc | Optional | Keyless JSON is often blocked (403). Shown as Unavailable | "Hot" ranking favours engaged posts. Engagement depends on subreddit size and culture. Comments only with PRAW. | |
| CoinGecko | CoinGecko | Market | None | Every 10 min | prices, 24h change, trending searches, global market | Collection time | Optional | Unavailable status. Reactions skip events without a price | Public rate limits. Trending reflects CoinGecko searches only. |
| Fear & Greed | alternative.me | Third-party composite | None | Every 10 min (updates daily) | value, label | Provider timestamp | Optional | Shown as unavailable (never a default value) | Overlaps with this product's inputs. External context only. |
| Google Trends | Google (unofficial endpoint) | Search attention | None | At most hourly | 7-day interest, 5 keywords | Fetch time (age shown) | Optional | Last cached values with their age, or unavailable | Often blocked from datacenter IPs. Relative index per term. |
| Funding and open interest | Binance futures public API | Derivatives context | None | Every 10 min | BTC/ETH funding, BTC OI change | Collection time | Optional | Unavailable status | Geo-blocked in some regions. One exchange only. Belongs to Derivatives Intelligence. |
| Economic calendar | Financial Modeling Prep | Macro catalysts | API key | Every 10 min | high-impact US/EU events in the next 48h | Provider time | Optional | Not configured / unavailable | Impact field is coarse. |
Duplication risk
| Path | Risk | Handling |
|---|---|---|
| Outlet RSS and the same article via Google News | High | Same publisher and headline = exact duplicate (dropped) |
| Wire or official story rewritten by many outlets | High | Story clustering. Counted as one story with N independent sources. |
| Press release republished across sites | High | Press-release markers collapse copies into one source |
| Sister sites (NewsBTC and Bitcoinist) | Medium | Shared ownership group |
| Reddit link post to an article | Medium | Joins the article's story as social discussion, not a source |
| Same Reddit post in several subreddits | Low | Separate posts. Each is rank-normalised within its subreddit. |
Credibility assumptions
Tiers and priors are listed above in the live table. They are judgement calls by the maintainers, reviewed in the v1 audit, and not validated against outcomes. The key change from pre-audit: Google News items used to carry a 1.2 weight, higher than most crypto outlets. They now carry the prior of the publisher that wrote them.