Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(analysis): better tolerate GIT analysis crashes #2884

Merged
merged 5 commits into from
Nov 21, 2024

Conversation

andrewpollock
Copy link
Contributor

@andrewpollock andrewpollock commented Nov 20, 2024

This commit improves tolerance for GIT commit analysis failures.

When GIT range analysis crashes, the entire record is discarded. The record's affected[] may contain other non-GIT events (i.e. package entries) that will be of value.

There are two known scenarios where GIT range analysis crashes:

  • the commit being analysed is an orphaned commit
  • the commit cannot be found in the repository (because it is from a fork)

Includes a test to confirm behaviour

This commit discards any GIT events when known failures in analysis are
seen, so that the rest of the record may still be imported successfully.

When GIT range analysis crashes, the entire record is discarded. The
record's `affected[]` may contain other non-GIT events (i.e. `package`
entries) that will be of value.

There are two known scenarios where GIT range analysis crashes:
- the commit being analysed is an orphaned commit
- the commit cannot be found in the repository (because it is from a
  fork)

Includes a test to ensure that invalid GIT events are removed from
records crashing analysis (and the record otherwise imports
successfully)
@andrewpollock
Copy link
Contributor Author

@pxp928 FYI

@andrewpollock
Copy link
Contributor Author

Once this is in Production, it should allow for the successful (partial) import of:

CVE-2016-10046
CVE-2016-10047
CVE-2016-10048
CVE-2016-10049
CVE-2016-10050
CVE-2016-10051
CVE-2016-10052
CVE-2016-10053
CVE-2016-10054
CVE-2016-10055
CVE-2016-10056
CVE-2016-10057
CVE-2016-10058
CVE-2016-10059
CVE-2016-10063
CVE-2016-10064
CVE-2016-10066
CVE-2016-10067
CVE-2016-10068
CVE-2016-10069
CVE-2016-10070
CVE-2016-10071
CVE-2016-10144
CVE-2016-10145
CVE-2016-10146
CVE-2016-10173
CVE-2016-10328
CVE-2016-1577
CVE-2016-1867
CVE-2016-1912
CVE-2016-2089
CVE-2016-2116
CVE-2016-2510
CVE-2016-4340
CVE-2016-4567
CVE-2016-4979
CVE-2016-5823
CVE-2016-5824
CVE-2016-5825
CVE-2016-5826
CVE-2016-5827
CVE-2016-7126
CVE-2016-7131
CVE-2016-7132
CVE-2016-7141
CVE-2016-7513
CVE-2016-7524
CVE-2016-7526
CVE-2016-7527
CVE-2016-7528
CVE-2016-7530
CVE-2016-7536
CVE-2016-7537
CVE-2016-7539
CVE-2016-7540
CVE-2016-8740
CVE-2016-9459
CVE-2016-9461
CVE-2016-9462
CVE-2016-9465
CVE-2016-9467
CVE-2016-9572
CVE-2016-9573
CVE-2016-9580
CVE-2016-9581
CVE-2017-0882
CVE-2017-11437
CVE-2017-11449
CVE-2017-11450
CVE-2017-11523
CVE-2017-11651
CVE-2017-12426
CVE-2017-13142
CVE-2017-13145
CVE-2017-13658
CVE-2017-14174
CVE-2017-15867
CVE-2017-15872
CVE-2017-16546
CVE-2017-17499
CVE-2017-4952
CVE-2017-5508
CVE-2017-5509
CVE-2017-5510
CVE-2017-5511
CVE-2017-5858
CVE-2017-5992
CVE-2017-6497
CVE-2017-6498
CVE-2017-6499
CVE-2017-6500
CVE-2017-6501
CVE-2017-6502
CVE-2017-7694
CVE-2017-9250
CVE-2017-9501
CVE-2018-10529
CVE-2018-14732
CVE-2018-14840
CVE-2018-14872
CVE-2018-14873
CVE-2018-16316
CVE-2018-16339
CVE-2018-19182
CVE-2018-20244
CVE-2018-5801
CVE-2018-6550
CVE-2018-6926
CVE-2018-8970
CVE-2018-9160
CVE-2019-0216
CVE-2019-0229
CVE-2019-11407
CVE-2019-11408
CVE-2019-11409
CVE-2019-11410
CVE-2019-12398
CVE-2019-12417
CVE-2019-12566
CVE-2019-12968
CVE-2019-13275
CVE-2019-13343
CVE-2019-14906
CVE-2019-15608
CVE-2019-16761
CVE-2019-16964
CVE-2019-16965
CVE-2019-16968
CVE-2019-16969
CVE-2019-16970
CVE-2019-16971
CVE-2019-16972
CVE-2019-16973
CVE-2019-16974
CVE-2019-16975
CVE-2019-16976
CVE-2019-16977
CVE-2019-16978
CVE-2019-16979
CVE-2019-16980
CVE-2019-16981
CVE-2019-16982
CVE-2019-16983
CVE-2019-16984
CVE-2019-16985
CVE-2019-16986
CVE-2019-16987
CVE-2019-16988
CVE-2019-16989
CVE-2019-16990
CVE-2019-16991
CVE-2019-19366
CVE-2019-19367
CVE-2019-19384
CVE-2019-19385
CVE-2019-19386
CVE-2019-19387
CVE-2019-19388
CVE-2019-20910
CVE-2019-20912
CVE-2019-20914
CVE-2019-6503
CVE-2020-11981
CVE-2020-11982
CVE-2020-11983
CVE-2020-14971
CVE-2020-15824
CVE-2020-19952
CVE-2020-20908
CVE-2020-21053
CVE-2020-21054
CVE-2020-21055
CVE-2020-21056
CVE-2020-21057
CVE-2020-24612
CVE-2020-24769
CVE-2020-24770
CVE-2020-24771
CVE-2020-28278
CVE-2020-28282
CVE-2020-8003
CVE-2020-9365
CVE-2020-9485
CVE-2021-21265
CVE-2021-25987
CVE-2021-28967
CVE-2021-30500
CVE-2021-30501
CVE-2021-35331
CVE-2021-3620
CVE-2021-3853
CVE-2021-3857
CVE-2021-3869
CVE-2021-3878
CVE-2021-3903
CVE-2021-39138
CVE-2021-39228
CVE-2021-3927
CVE-2021-3928
CVE-2021-3968
CVE-2021-3973
CVE-2021-3974
CVE-2021-3984
CVE-2021-4020
CVE-2021-40532
CVE-2021-4069
CVE-2021-4136
CVE-2021-4146
CVE-2021-4166
CVE-2021-4192
CVE-2021-4193
CVE-2021-43679
CVE-2021-44584
CVE-2022-0156
CVE-2022-0158
CVE-2022-0198
CVE-2022-0213
CVE-2022-0239
CVE-2022-0256
CVE-2022-0257
CVE-2022-0258
CVE-2022-0260
CVE-2022-0262
CVE-2022-0318
CVE-2022-0319
CVE-2022-0368
CVE-2022-0393
CVE-2022-0407
CVE-2022-0408
CVE-2022-0413
CVE-2022-0443
CVE-2022-0554
CVE-2022-0629
CVE-2022-0696
CVE-2022-0714
CVE-2022-0729
CVE-2022-0943
CVE-2022-1420
CVE-2022-1429
CVE-2022-1554
CVE-2022-1619
CVE-2022-1620
CVE-2022-1621
CVE-2022-1629
CVE-2022-1674
CVE-2022-1720
CVE-2022-1725
CVE-2022-1726
CVE-2022-1733
CVE-2022-1735
CVE-2022-1769
CVE-2022-1771
CVE-2022-1796
CVE-2022-1851
CVE-2022-1886
CVE-2022-1898
CVE-2022-1927
CVE-2022-1968
CVE-2022-2042
CVE-2022-2054
CVE-2022-2079
CVE-2022-2124
CVE-2022-2125
CVE-2022-2126
CVE-2022-2175
CVE-2022-2182
CVE-2022-2183
CVE-2022-2206
CVE-2022-2207
CVE-2022-2208
CVE-2022-2210
CVE-2022-22126
CVE-2022-2231
CVE-2022-2257
CVE-2022-2264
CVE-2022-2284
CVE-2022-2285
CVE-2022-2286
CVE-2022-2287
CVE-2022-2289
CVE-2022-23053
CVE-2022-23054
CVE-2022-2343
CVE-2022-2344
CVE-2022-2345
CVE-2022-24883
CVE-2022-2522
CVE-2022-2571
CVE-2022-2580
CVE-2022-2581
CVE-2022-2598
CVE-2022-2631
CVE-2022-2817
CVE-2022-2819
CVE-2022-2845
CVE-2022-2849
CVE-2022-2862
CVE-2022-2874
CVE-2022-2889
CVE-2022-2922
CVE-2022-2923
CVE-2022-2925
CVE-2022-2946
CVE-2022-2980
CVE-2022-2982
CVE-2022-3016
CVE-2022-3019
CVE-2022-3037
CVE-2022-31372
CVE-2022-3153
CVE-2022-31783
CVE-2022-32096
CVE-2022-32323
CVE-2022-3234
CVE-2022-3235
CVE-2022-32442
CVE-2022-32444
CVE-2022-3255
CVE-2022-3256
CVE-2022-3278
CVE-2022-3296
CVE-2022-3297
CVE-2022-3348
CVE-2022-3352
CVE-2022-3491
CVE-2022-34937
CVE-2022-3570
CVE-2022-3591
CVE-2022-36040
CVE-2022-36041
CVE-2022-36043
CVE-2022-3705
CVE-2022-3827
CVE-2022-38493
CVE-2022-39243
CVE-2022-40443
CVE-2022-40444
CVE-2022-40446
CVE-2022-40447
CVE-2022-4069
CVE-2022-4070
CVE-2022-4111
CVE-2022-41945
CVE-2022-4292
CVE-2022-44361
CVE-2022-45801
CVE-2022-46365
CVE-2022-46609
CVE-2022-47015
CVE-2022-47024
CVE-2023-0049
CVE-2023-0051
CVE-2023-0288
CVE-2023-0433
CVE-2023-22487
CVE-2023-22488
CVE-2023-22489
CVE-2023-26043
CVE-2023-26125
CVE-2023-30258
CVE-2023-39344
CVE-2023-40171
CVE-2023-43877
CVE-2023-43878
CVE-2023-43879
CVE-2023-44487
CVE-2023-44767
CVE-2023-47635
CVE-2023-50020
CVE-2023-6940
CVE-2024-0875
CVE-2024-1183
CVE-2024-23647
CVE-2024-29022
CVE-2024-29023
CVE-2024-2914
CVE-2024-2928
CVE-2024-3098
CVE-2024-3271
CVE-2024-34063
CVE-2024-34353
CVE-2024-40640
CVE-2024-4181
CVE-2024-4263
CVE-2024-4343
CVE-2024-45118
CVE-2024-5213

This commit acts on reviewer feedback and:
- treats individual instances of invalid commits as invalid
  - Orphaned commits are treated the same as other classes of invalid
    commits
  - Commits not found in a branch are handled more gracefully
- Entire GIT ranges are no longer excluded as a result of this
This commit reverts an unnecessary change to reduce the size of the diff
It also corrects the test docstring to match the newer reality
osv/impact.py Outdated Show resolved Hide resolved
A different approach was taken
@andrewpollock andrewpollock merged commit 743b45e into google:master Nov 21, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants