Skip to content

extractus/feed-extractor

Repository files navigation

feed-reader

To read & normalize RSS/ATOM/JSON feed data.

NPM CI test Coverage Status CodeQL JavaScript Style Guide

Deploy

Demo

Usage

import { read } from 'feed-reader'

// with CommonJS environments
// const { read } = require('feed-reader/dist/cjs/feed-reader.js')

const url = 'https://news.google.com/rss'

read(url).then((feed) => {
  console.log(feed)
}).catch((err) => {
  console.log(err)
})

APIs

read(String url)

Load and extract feed data from given RSS/ATOM/JSON source. Return a Promise object.

Example:

import {
  read
} from 'feed-reader'

const getFeedData = async (url) => {
  try {
    console.log(`Get feed data from ${url}`)
    const result = await read(url)
    // result may be feed data or null
    console.log(result)
    return result
  } catch (err) {
    console.trace(err)
  }
}

getFeedData('https://news.google.com/rss')
getFeedData('https://news.google.com/atom')
getFeedData('https://adactio.com/journal/feed.json')

Feed data object retuned by read() method should look like below:

{
  "title": "Top stories - Google News",
  "link": "https://news.google.com/atom?hl=en-US&gl=US&ceid=US%3Aen",
  "description": "Google News",
  "generator": "NFE/5.0",
  "language": "",
  "published": "2021-12-23T15:01:12.000Z",
  "entries": [
    {
      "title": "Lone suspect in Waukesha parade crash to appear in court today, as Wisconsin reels from tragedy that left 5 dead and dozens more injured - CNN",
      "link": "https://news.google.com/__i/rss/rd/articles/CBMiTmh0dHBzOi8vd3d3LmNubi5jb20vMjAyMS8xMS8yMy91cy93YXVrZXNoYS1jYXItcGFyYWRlLWNyb3dkLXR1ZXNkYXkvaW5kZXguaHRtbNIBUmh0dHBzOi8vYW1wLmNubi5jb20vY25uLzIwMjEvMTEvMjMvdXMvd2F1a2VzaGEtY2FyLXBhcmFkZS1jcm93ZC10dWVzZGF5L2luZGV4Lmh0bWw?oc=5",
      "description": "Lone suspect in Waukesha parade crash to appear in court today, as Wisconsin reels from tragedy that left 5 dead and dozens more injured    CNN Waukesha Christmas parade attack: 5 dead, 48 injured, Darrell Brooks named as...",
      "published": "2021-12-21T22:30:00.000Z"
    },
    // ...
  ]
}

Configuration methods

setRequestOptions(Object requestOptions)

Affect to the way how axios works. Please refer axios' request config for more info.

getRequestOptions()

Return current request options.

Default values can be found here.

setReaderOptions(Object readerOptions)

To change default reader options.

  • descriptionMaxLen: Number, max num of chars for description (default: 210)
  • includeFullContent: Boolean, add content to entry if available (default: false)
  • convertPubDateToISO: Boolean, reformat published date to ISO standard (default: true)

getReaderOptions()

Return current reader options.

Test

git clone https://github.com/ndaidong/feed-reader.git
cd feed-reader
npm install

# quick evaluation
npm run eval https://news.google.com/rss
npm test

License

The MIT License (MIT)