|
| 1 | +======================= |
| 2 | +VulnerableCode Insights |
| 3 | +======================= |
| 4 | + |
| 5 | +| Mentee: **Sampurna Pyne** |
| 6 | +| GitHub: `Samk1710 <https://github.com/Samk1710>`_ |
| 7 | +| LinkedIn: `@samk1710 <https://www.linkedin.com/in/samk1710/>`_ |
| 8 | +| Repository: `VulnerableCode <https://github.com/aboutcode-org/vulnerablecode>`_ |
| 9 | +| Official GSoC project page: `Project Link |
| 10 | + <https://summerofcode.withgoogle.com/programs/2026/projects/nlwOImQP>`_ |
| 11 | +| GSoC Proposal: `Proposal Link |
| 12 | + <https://docs.google.com/document/d/1xBapGNBjmkRrNeKFeXPyB014XOkDcpbJNFjGuw59sAc/edit?tab=t.0>`_ |
| 13 | +
|
| 14 | +Overview |
| 15 | +-------- |
| 16 | + |
| 17 | +VulnerableCode imports vulnerability advisories from dozens of data sources and |
| 18 | +grows its database continuously. However, it previously lacked a way to |
| 19 | +demonstrate its quality. This project enables VulnerableCode to visually |
| 20 | +showcase its data richness and coverage, while highlighting areas for improvement. This transparency |
| 21 | +gives users confidence in VulnerableCode and provides maintainers with actionable |
| 22 | +insights to improve it. |
| 23 | + |
| 24 | +.. raw:: html |
| 25 | + |
| 26 | + <div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; height: auto; margin-bottom: 2em; margin-top: 1em;"> |
| 27 | + <iframe src="https://www.youtube.com/embed/pCTos7DMD5c" frameborder="0" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe> |
| 28 | + </div> |
| 29 | + |
| 30 | +This GSoC project builds three key features: |
| 31 | + |
| 32 | +* `Insights Dashboard`_: A multi-panel dashboard containing various interactive |
| 33 | + charts to showcase VulnerableCode's strengths and identify areas for improvement. |
| 34 | +* `History of Advisories`_: A diff view for tracking changes in an advisory and |
| 35 | + viewing its historical versions as snapshots. |
| 36 | +* `EPSS History`_: Visual tracking of EPSS score and percentile trends over time. |
| 37 | + |
| 38 | +Detailed Report |
| 39 | +--------------- |
| 40 | + |
| 41 | +Insights Dashboard |
| 42 | +^^^^^^^^^^^^^^^^^^ |
| 43 | + |
| 44 | +The Insights Dashboard is a multi-panel interactive dashboard designed to |
| 45 | +visualize various features of VulnerableCode through infographic charts. |
| 46 | + |
| 47 | +A dedicated ``insights`` Django app houses the dashboard's core functionality. |
| 48 | +To ensure fast and scalable chart rendering, an automated daily snapshot pipeline |
| 49 | +(``insights_snapshot_pipeline.py``) pre-computes database-wide metrics once every night, avoiding |
| 50 | +heavy on the fly queries across VulnerableCode's million-row tables. These metrics |
| 51 | +are stored in dedicated models linked to a ``DailySnapshot``, keeping the telemetry |
| 52 | +structured and instantly queryable. |
| 53 | + |
| 54 | +Below is the tree view of ``insights`` for better understanding :: |
| 55 | + |
| 56 | + insights |
| 57 | + ├── models.py |
| 58 | + ├── views.py |
| 59 | + ├── urls.py |
| 60 | + ├── utils.py |
| 61 | + ├── insights_snapshot_pipeline.py |
| 62 | + ├── charts/ |
| 63 | + │ ├── __init__.py |
| 64 | + │ ├── overview_panel.py |
| 65 | + │ ├── package_panel.py |
| 66 | + │ ├── severity_panel.py |
| 67 | + │ ├── importer_panel.py |
| 68 | + │ └── data_quality_panel.py |
| 69 | + ├── templates/insights/ |
| 70 | + │ ├── dashboard.html |
| 71 | + │ └── components/ |
| 72 | + └── static/insights/ |
| 73 | + ├── css/insights.css |
| 74 | + └── js/ |
| 75 | + |
| 76 | + |
| 77 | +The dashboard panels are implemented using a modular framework in the ``charts/`` |
| 78 | +directory. Each chart is driven by a ``ChartDefinition`` dataclass, which encapsulates |
| 79 | +all the metadata, collection, and formatting logic required to render it. These |
| 80 | +definitions are centrally registered in the ``insights/charts/__init__.py`` chart registry. |
| 81 | +At request time, the API simply fetches the latest pre-computed snapshot using these |
| 82 | +definitions, ensuring instantaneous page loads. |
| 83 | + |
| 84 | +The UI integrates Billboard.js to render interactive charts. |
| 85 | + |
| 86 | +| More: |
| 87 | +| https://github.com/aboutcode-org/vulnerablecode/pull/2391 |
| 88 | +| https://github.com/aboutcode-org/vulnerablecode/pull/2408 |
| 89 | +
|
| 90 | + |
| 91 | +History of Advisories |
| 92 | +^^^^^^^^^^^^^^^^^^^^^ |
| 93 | + |
| 94 | +Advisory data imported from upstream sources is constantly updated. Previously, |
| 95 | +VulnerableCode only exposed the latest version of an advisory, making |
| 96 | +it impossible for users to track what changed over time. |
| 97 | + |
| 98 | +This feature introduces a chronological list of advisory versions, allowing users |
| 99 | +to click into any historical snapshot and view the advisory exactly as it |
| 100 | +existed at that moment. To provide a quick overview, the UI displays diffs between |
| 101 | +versions directly. |
| 102 | + |
| 103 | +These diffs are generated at import time. For existing advisories, |
| 104 | +they can also be backfilled using the ``HistoryDiffImproverPipeline``. The diffs |
| 105 | +are stored in the ``AdvisoryHistoryDiff`` model. |
| 106 | + |
| 107 | + |
| 108 | +| More: https://github.com/aboutcode-org/vulnerablecode/pull/2356 |
| 109 | +
|
| 110 | + |
| 111 | + |
| 112 | + |
| 113 | +EPSS History |
| 114 | +^^^^^^^^^^^^ |
| 115 | + |
| 116 | +This feature adds an EPSS History Trend on the advisory page in the form of a line chart |
| 117 | +and a paginated history table showing the percentile and score of EPSS over time |
| 118 | +for a given advisory. |
| 119 | + |
| 120 | +| More: https://github.com/aboutcode-org/vulnerablecode/pull/2328 |
| 121 | +
|
| 122 | + |
| 123 | + |
| 124 | + |
| 125 | +Linked Pull Requests |
| 126 | +-------------------- |
| 127 | + |
| 128 | +.. list-table:: |
| 129 | + :widths: 10 60 30 |
| 130 | + :width: 100% |
| 131 | + :header-rows: 1 |
| 132 | + |
| 133 | + * - No. |
| 134 | + - Name |
| 135 | + - Link |
| 136 | + * - 1 |
| 137 | + - Insights Dashboard (Part 1) |
| 138 | + - `vulnerablecode#2391 <https://github.com/aboutcode-org/vulnerablecode/pull/2391>`_ |
| 139 | + * - 2 |
| 140 | + - Insights Dashboard (Part 2) |
| 141 | + - `vulnerablecode#2408 <https://github.com/aboutcode-org/vulnerablecode/pull/2408>`_ |
| 142 | + * - 3 |
| 143 | + - History of Advisories |
| 144 | + - `vulnerablecode#2356 <https://github.com/aboutcode-org/vulnerablecode/pull/2356>`_ |
| 145 | + * - 4 |
| 146 | + - EPSS History |
| 147 | + - `vulnerablecode#2328 <https://github.com/aboutcode-org/vulnerablecode/pull/2328>`_ |
| 148 | + |
| 149 | + |
| 150 | +Pre-GSoC Work |
| 151 | +------------- |
| 152 | + |
| 153 | +.. list-table:: |
| 154 | + :widths: 10 60 30 |
| 155 | + :width: 100% |
| 156 | + :header-rows: 1 |
| 157 | + |
| 158 | + * - No. |
| 159 | + - Name |
| 160 | + - Link |
| 161 | + * - 1 |
| 162 | + - Add Mirror Pipeline for EUVD using GitHub Actions |
| 163 | + - `aboutcode-mirror-euvd#1 <https://github.com/aboutcode-org/aboutcode-mirror-euvd/pull/1>`_ |
| 164 | + * - 2 |
| 165 | + - Document the readme.md for EUVD Mirror |
| 166 | + - `aboutcode-mirror-euvd#3 <https://github.com/aboutcode-org/aboutcode-mirror-euvd/pull/3>`_ |
| 167 | + * - 3 |
| 168 | + - Add Importer Pipeline for Tuxcare Advisories |
| 169 | + - `vulnerablecode#2104 <https://github.com/aboutcode-org/vulnerablecode/pull/2104>`_ |
| 170 | + * - 4 |
| 171 | + - Add Importer Pipeline for Vmware Photon Advisories |
| 172 | + - `vulnerablecode#2198 <https://github.com/aboutcode-org/vulnerablecode/pull/2198>`_ |
| 173 | + * - 5 |
| 174 | + - Add Importer Pipeline for Openstack Advisories |
| 175 | + - `vulnerablecode#2154 <https://github.com/aboutcode-org/vulnerablecode/pull/2154>`_ |
| 176 | + * - 6 |
| 177 | + - Fix Nix Flake error in GitHub Actions |
| 178 | + - `vulnerablecode#2161 <https://github.com/aboutcode-org/vulnerablecode/pull/2161>`_ |
| 179 | + * - 7 |
| 180 | + - Add Importer Pipeline for EUVD |
| 181 | + - `vulnerablecode#2046 <https://github.com/aboutcode-org/vulnerablecode/pull/2046>`_ |
| 182 | + * - 8 |
| 183 | + - Add alpine in RANGE_CLASS_BY_SCHEME |
| 184 | + - `univers#185 <https://github.com/aboutcode-org/univers/pull/185>`_ |
| 185 | + * - 9 |
| 186 | + - Add support for Openstack Advisories |
| 187 | + - `univers#184 <https://github.com/aboutcode-org/univers/pull/184>`_ |
| 188 | + * - 10 |
| 189 | + - Fix missing cwe2 dependency |
| 190 | + - `vulnerablecode-ai-experiments#15 <https://github.com/aboutcode-org/vulnerablecode-ai-experiments/pull/15>`_ |
| 191 | + * - 11 |
| 192 | + - Identify and fix env load failures due to improper typecasts |
| 193 | + - `vulnerablecode-ai-experiments#18 <https://github.com/aboutcode-org/vulnerablecode-ai-experiments/pull/18>`_ |
| 194 | + * - 12 |
| 195 | + - Researched and documented VulDB API for Vulntotal |
| 196 | + - `vulnerablecode#1199 <https://github.com/aboutcode-org/vulnerablecode/issues/1199>`_ |
| 197 | + |
| 198 | + |
| 199 | +Post-GSoC |
| 200 | +--------- |
| 201 | + |
| 202 | +The Insights Dashboard can be extended further with additional charts and panels to provide even |
| 203 | +richer data analytics and visibility. These insights act as a direct feedback loop, helping us |
| 204 | +continuously refine VulnerableCode by guiding the development of new data pipelines, |
| 205 | +targeted improvers, and architectural enhancements. |
| 206 | + |
| 207 | + |
| 208 | +Closing Thoughts |
| 209 | +---------------- |
| 210 | + |
| 211 | +Working on VulnerableCode Insights was a truly rewarding experience that I thoroughly enjoyed. |
| 212 | +Designing the Insights Dashboard was a particularly thought-intensive process where |
| 213 | +I found myself drawing inspiration from various insights features like Spotify Capsule |
| 214 | +and GitHub Insights. |
| 215 | + |
| 216 | +The most amazing part of this journey was the `weekly meetings <https://meet.jit.si/AboutCode>`_ with my mentors. It was an |
| 217 | +incredibly smooth experience, and I owe a huge thanks to them for all their |
| 218 | +thoughtful inputs, guidance and feedback: |
| 219 | + |
| 220 | +Mentors: `Philippe Ombredanne <https://github.com/pombredanne>`_, `Hritik Vijay <https://github.com/hritik14>`_, `Keshav Priyadarshi <https://github.com/keshav-space>`_, `Ziad Hany <https://github.com/ziadhany>`_, `Ayan Sinha Mahapatra <https://github.com/AyanSinhaMahapatra>`_, and `Tushar Goel <https://github.com/TG1999>`_ |
| 221 | + |
| 222 | +It was truly amazing to have this opportunity to learn, contribute, and grow with `AboutCode <https://github.com/aboutcode-org>`_ this summer, filled with tons of bugs and caffeine. |
| 223 | + |
| 224 | +Until next time! |
0 commit comments