Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: extractus/feed-extractor
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v5.0.0rc1
Choose a base ref
...
head repository: extractus/feed-extractor
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Loading
Showing with 6,045 additions and 6,128 deletions.
  1. +21 −10 .github/workflows/ci-test.yml
  2. +70 −0 .github/workflows/codeql-analysis.yml
  3. +4 −0 .gitignore
  4. +6 −0 .npmignore
  5. +71 −0 CONTRIBUTING.md
  6. +1 −1 LICENSE
  7. +335 −77 README.md
  8. +17 −0 SECURITY.md
  9. +18 −51 build.js
  10. +0 −25 build.test.js
  11. +0 −24 cjs-eval.js
  12. +0 −5,310 dist/cjs/feed-reader.js
  13. +0 −5 dist/cjs/package.json
  14. +129 −0 eslint.config.js
  15. +10 −23 eval.js
  16. +17 −0 examples/browser-feed-reader/.gitignore
  17. +30 −0 examples/browser-feed-reader/README.md
  18. +12 −0 examples/browser-feed-reader/package.json
  19. +1 −0 examples/browser-feed-reader/public/chota.min.css
  20. +88 −0 examples/browser-feed-reader/public/index.html
  21. +30 −0 examples/browser-feed-reader/server.js
  22. +171 −0 examples/bun-feed-reader/.gitignore
  23. +31 −0 examples/bun-feed-reader/README.md
  24. +50 −0 examples/bun-feed-reader/index.ts
  25. +14 −0 examples/bun-feed-reader/package.json
  26. +9 −0 examples/bun-feed-reader/tsconfig.json
  27. +29 −0 examples/deno-feed-reader/README.md
  28. +13 −0 examples/deno-feed-reader/deno.json
  29. +52 −0 examples/deno-feed-reader/index.ts
  30. +17 −0 examples/node-feed-reader/.gitignore
  31. +33 −0 examples/node-feed-reader/README.md
  32. +49 −0 examples/node-feed-reader/index.js
  33. +13 −0 examples/node-feed-reader/package.json
  34. +19 −0 examples/tsnode-feed-reader/.gitignore
  35. +37 −0 examples/tsnode-feed-reader/README.md
  36. +49 −0 examples/tsnode-feed-reader/index.ts
  37. +17 −0 examples/tsnode-feed-reader/package.json
  38. +11 −0 examples/tsnode-feed-reader/tsconfig.json
  39. +90 −9 index.d.ts
  40. +38 −37 package.json
  41. +3 −6 reset.js
  42. +14 −0 src/cjs-entry.js
  43. +0 −21 src/config.js
  44. +0 −26 src/config.test.js
  45. +2 −0 src/deno/cross-fetch.js
  46. +70 −31 src/main.js
  47. +470 −93 src/main.test.js
  48. +0 −9 src/utils/isValidUrl.js
  49. +22 −4 src/utils/{purifyUrl.js → linker.js}
  50. +138 −0 src/utils/linker.test.js
  51. +0 −15 src/utils/logger.js
  52. +113 −0 src/utils/normalizer.js
  53. +13 −0 src/utils/normalizer.test.js
  54. +140 −0 src/utils/parseAtomFeed.js
  55. +89 −0 src/utils/parseJsonFeed.js
  56. +129 −0 src/utils/parseRdfFeed.js
  57. +144 −0 src/utils/parseRssFeed.js
  58. +0 −124 src/utils/parser.js
  59. +0 −54 src/utils/purifyUrl.test.js
  60. +61 −18 src/utils/retrieve.js
  61. +56 −27 src/utils/retrieve.test.js
  62. +0 −17 src/utils/validator.js
  63. +0 −34 src/utils/validator.test.js
  64. +0 −15 src/utils/xml2obj.js
  65. +32 −0 src/utils/xmlparser.js
  66. +41 −0 src/utils/xmlparser.test.js
  67. +283 −0 test-data/atom-feed-standard-realworld.xml
  68. +30 −0 test-data/atom-feed-standard.xml
  69. +5 −1 test-data/{another-atom.xml → atom-multilinks.xml}
  70. +0 −31 test-data/atom.xml
  71. +23 −0 test-data/json-feed-miss-base-url.json
  72. +233 −0 test-data/json-feed-standard-realworld.json
  73. +23 −0 test-data/json-feed-standard.json
  74. +56 −0 test-data/medium-feed.xml
  75. +33 −0 test-data/podcast.rss
  76. +108 −0 test-data/rdf-standard.xml
  77. +885 −0 test-data/rss-feed-miss-base-url.xml
  78. +885 −0 test-data/rss-feed-miss-link.xml
  79. +318 −0 test-data/rss-feed-standard-realworld.xml
  80. +24 −0 test-data/rss-feed-standard.xml
  81. +0 −30 test-data/rss.xml
31 changes: 21 additions & 10 deletions .github/workflows/ci-test.yml
Original file line number Diff line number Diff line change
@@ -8,40 +8,51 @@ on: [push, pull_request]
jobs:
test:

runs-on: ubuntu-20.04
runs-on: ubuntu-latest

strategy:
matrix:
node_version: [14.x, 15.x, 16.x, 17.x]
node_version: [18.x, 20.x, 22.x]

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

- name: setup Node.js v${{ matrix.node_version }}
uses: actions/setup-node@v1
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node_version }}

- name: run npm scripts
env:
PROXY_SERVER: ${{ secrets.PROXY_SERVER }}
run: |
npm i -g standard
npm install
npm run lint
npm run build --if-present
npm run test
- name: sync to coveralls
uses: coverallsapp/github-action@v1.1.2
- name: Coveralls Parallel
uses: coverallsapp/github-action@v2
with:
flag-name: run-${{ join(matrix.*, '-') }}
parallel: true
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: cache node modules
uses: actions/cache@v2
uses: actions/cache@v4
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
finish:
needs: test
if: ${{ always() }}
runs-on: ubuntu-latest
steps:
- name: Coveralls Finished
uses: coverallsapp/github-action@v2
with:
parallel-finished: true
carryforward: "run-18.x,run-20.x,run-21.x"


70 changes: 70 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
push:
branches: [ main ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ main ]
schedule:
- cron: '33 18 * * 3'

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write

strategy:
fail-fast: false
matrix:
language: [ 'javascript' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
- name: Checkout repository
uses: actions/checkout@v4

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main

# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2

# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun

# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language

#- run: |
# make bootstrap
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -17,3 +17,7 @@ package-lock.json
pnpm-lock.yaml

output.json
deno.lock

bundle.cjs
bundle.cjs.map
6 changes: 6 additions & 0 deletions .npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
node_modules
coverage
.github
pnpm-lock.yaml
examples
test-data
71 changes: 71 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Contributing to `@extractus/feed-extractor`

Glad to see you here.

Collaborations and pull requests are always welcomed, though larger proposals should be discussed first.

As an OSS, it's better to follow the Unix philosophy: "do one thing and do it well".


## Third-party libraries

Please avoid using libaries other than those available in the standard library, unless necessary.

This library needs to be simple and flexible to run on multiple platforms such as Deno, Bun, or even browser.


## Coding convention

Make sure your code lints before opening a pull request.

```bash
cd feed-extractor

# check coding convention issue
npm run lint

# auto fix coding convention issue
npm run lint:fix
```

*When you run `npm test`, the linting process will be triggered at first.*


## Testing

Be sure to run the unit test suite before opening a pull request. An example test run is shown below.

```bash
cd feed-extractor
npm test
```

![feed-extractor unit test](https://i.imgur.com/2b5xt6S.png)

If test coverage decreased, please check test scripts and try to improve this number.


## Documentation

If you've changed APIs, please update README and [the examples](examples).


## Clean commit histories

When you open a pull request, please ensure the commit history is clean.
Squash the commits into logical blocks, perhaps a single commit if that makes sense.

What you want to avoid is commits such as "WIP" and "fix test" in the history.
This is so we keep history on master clean and straightforward.

For people new to git, please refer the following guides:

- [Writing good commit messages](https://github.com/erlang/otp/wiki/writing-good-commit-messages)
- [Commit Message Guidelines](https://gist.github.com/robertpainsi/b632364184e70900af4ab688decf6f53)


## License

By contributing to `@extractus/feed-extractor`, you agree that your contributions will be licensed under its [MIT license](LICENSE).

---
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
The MIT License (MIT)

Copyright (c) 2015 Dong Nguyen
Copyright (c) 2015 Extractus

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Loading