Issue980 - Add verifyHeader method to prevent erronous headers from passing through

[andreleblanc11]

There is one commit that has the wrong issue number.. oops. I typed the wrong issue number in the commit. I also tried reverting the change through this thread https://docs.github.com/en/pull-requests/committing-changes-to-your-project/creating-and-editing-commits/changing-a-commit-message#amending-older-or-multiple-commit-messages but that also didn't work :(

The verifyHeader method is ported from Sundew and checks if the header contents are what they're supposed to be.

I added the isProblem to the FlowCB renamer so that we can keep saving the problematic bulletins locally when they are flagged from verifyHeader

For testing

• I've been running a separate sr3 AM server on my DIRT VM with a connection through the Edmonton regional (our most popular one), and have been monitoring the file output. So far it's been looking good. The bad bulletins now also get an extra layer of error messages when there header is malformed.

2024-04-30 00:01:32,936 [DEBUG] sarracenia.flowcb.gather.am gather Bulletin contents: b'CA\nWPK\n1.96225,140.334,21.8998\n'
2024-04-30 00:01:32,936 [ERROR] sarracenia.bulletin verifyHeader Incomplete header (less than 3 fields)
2024-04-30 00:01:32,936 [DEBUG] sarracenia.flowcb.gather.am correctContents Missing contents added
2024-04-30 00:01:32,936 [ERROR] sarracenia.flowcb.rename.raw2bulletin rename Unable to get julian time.
2024-04-30 00:01:32,936 [ERROR] sarracenia.flowcb.rename.raw2bulletin rename New filename (for problem file): CACN00_CWAO_300001__WPK_00635_PROBLEM

• I tested the erroneous file that caused this method to be added and it's now being dealt with properly.
• I also didn't want to test all AM regional feeds because we would still be missing the DND ones and it would make it hard to accurately compare with the data that's already on NCP

[github-actions]

Test Results

209 tests   201 ✅  17s ⏱️
  1 suites    8 💤
  1 files      0 ❌

Results for commit 6fb14a9.

♻️ This comment has been updated with latest results.

[petersilva]

You set assignees... not reviewers... I think I've done that too in the past...

[petersilva]

I guess if the logic comes from Sundew, it's been around for 20 years... it's got to be ok.

[andreleblanc11]

Flakey broker flow test not passing is expected. Fixed with #1032