Issue1029 wmo00 accumulated file format support

[petersilva]

WMO-00 is a format for data exchange with WMO GTS nodes.

One needs to put a header and possibly a trailer on the content of inbound individual files. Once the headers are added the data for multiple messages are catenated together. The resulting accumulation file is then named CCCC00000000.a where the CCCC is the WMO origin code, the 8 digits are a numbers to make the files unique, and the a suffix is a type specifier.

There are two plugins introduced to support:

• filter.wmo00_accumulate -- given a flow of individual WMO messages create the accumulated files.
• filter.wmo00_split -- given a flow of accumulated files, create files each containing one WMO message.

Reference: https://library.wmo.int/viewer/35800/download?file=386_en_1.pdf&type=pdf&navigator=1
( Page 137... the accumulated message format.)

Describing the patches:

So

• sarracenia.message.getContent() has been improved so that it works with files that are on the local machine.
  You don't have to calculate the file name and open it. just call getContent() it will lookup the baseDir setting,
  and check if such a file exists locally.
• message.getContent() was also improved to work with file:/ urls (only worked with http/https before.
• the two plugins.
• robustification: some fixes to mainline code resulting from messages with different and options fields being omiteed.
• was getting annoyed by having to run stop after foreground... turned out it was not checking for missing.
• the rest is just the result of attempted rebasing from development... apparently starting point just was not uptodate.

How it was tested:

set up subscription to WMO-BULLETINS on a test marchine, then have the two modules called

    callback filter.wmo00_accumulate
    callback filter.wmo00_split

This provides a round-tripping test. During development ran with messageCountMax 50 to see short runs. Looking at the logs, could determine if the files were coming out of the grouped file intact. Also, since the file
naming cannot match (using randomized sequence numbers) applied md5 checksums to make
correlation of input and output easier.

Once it was working, ran it for a few hours on a full international WMO GTS feed.
looking for errors. So the two are at least consistent with each other.

Used international WMO bulletins present in the subscription as examples to better understand the formats.

next step would be interop tests, but probably need it merged first.

EDIT: changed names of modules from wmo00_write -> wmo00_accumulate, wmo00_read -> wmo00_split.

[github-actions]

Test Results

209 tests  ±0   201 ✅ ±0   16s ⏱️ -2s
  1 suites ±0     8 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 6cba815. ± Comparison against base commit 2851416.

♻️ This comment has been updated with latest results.

[petersilva]

config settings:

subtopic *.SSC-DATAINTERCHANGE.WMO-BULLETINS.#

baseDir  /apps/sarra/public_data

wmo00_work_directory /tmp/wmo00
wmo00_origin_CCCC CWAO
wmo00_encapsulate on

callback filter.wmo00_accumulate
callback filter.wmo00_split

expire 300
messageCountMax 10
batch 5
#batch 100
post_format v02

accept .*

sample run:
wmo00_run.log

UPDATE: module names changed... the attachment is a run from before the change was made so modules have the old names.

[petersilva]

fwiw... I call it WMO-00 format because I heard that somewhere (possibly in wmo meetings.) I think it refers to bytes 9 and 10 in the header... which are 0's. (aka nulls.) There is another type described on the same page, where the type specifier is 01 ... but that file format is deprecated according to the source, and our French colleagues.

[petersilva]

questions:

• there is wmo00_work_directory which is where the output files are written.
  Should it just use self.o.baseDirectory when writing accumulated files?
  I think the accumulated files should not be kept around their names are not useful, and people likely won't want them, they are just temporary files to feed to another server. The individual files are much more useful.

• Similarly, when extracting (with wmo00_read) from the accumulated file, we write into wmo00_work_directory... is that good? Those are files that we would expect people to subscribe to , so perhaps "directory" is maybe a good root?

I'm scarred by amtcpwriter... ;-) where nobody knows what _read and _write are referring to. perhaps different names for the modules.... suggestions?
wmo-00 definitely refers to the accumulated files, not the individual messages.

• wmo00_write becomes wmo00_accumulate (which builds accumulations of individual files.)
• wmo00_read becomes wmo00_separate (parse accumulations into individual files.)
  or wmo00_extract ?

Would names like that be clearer?

[github-actions]

Coverage Report •

File	Stmts	Miss	Cover	Missing
__init__.py	416	46	88%	70–71, 73, 80, 83, 87–88, 90–91, 98, 104, 107, 109–110, 112–113, 115–116, 320, 431, 464, 531–532, 561–562, 608, 621, 623, 633–635, 645–646, 675, 684, 780, 813, 829, 883–885, 888, 891–893, 899
config.py	1415	1070	24%	38, 40–43, 68, 74, 328–330, 334, 338, 346, 348–350, 362–363, 365–366, 368, 370, 377, 379–380, 382, 384–385, 387, 393, 395–396, 398, 400–401, 403, 406, 408–414, 416–422, 424, 426, 435–441, 443–447, 449, 451, 453–455, 457, 459–461, 463–465, 467, 469–470, 472–474, 476, 478–479, 481–484, 486–487, 489, 498, 500, 504–509, 511, 516–517, 519, 523–530, 533–534, 538, 540, 542–543, 547, 549, 551–552, 556–557, 559–561, 564–567, 569, 771–772, 840–847, 849, 851–852, 869, 875–886, 890, 894–897, 899, 903, 908–910, 916, 923, 925, 941, 946, 949, 956–957, 960, 965–969, 972–974, 980–984, 986–987, 989–992, 994–995, 997–998, 1006, 1008–1014, 1022, 1025–1028, 1030–1032, 1034, 1036–1037, 1039, 1041–1044, 1082–1083, 1089, 1091–1093, 1095–1097, 1099–1101, 1110, 1112–1114, 1123–1125, 1132, 1134–1135, 1143, 1148–1156, 1158, 1160–1161, 1163–1165, 1167–1168, 1174, 1176–1177, 1179–1180, 1182–1183, 1185, 1191–1192, 1196–1197, 1199–1204, 1206, 1211–1212, 1214–1215, 1225–1227, 1229–1230, 1234, 1252–1253, 1260, 1264–1265, 1267, 1269–1270, 1272, 1274, 1282–1283, 1285–1288, 1290–1291, 1293, 1295–1296, 1303–1304, 1306, 1308–1309, 1311, 1315–1324, 1345–1348, 1350, 1355–1358, 1360, 1362, 1364, 1367–1372, 1374, 1377–1378, 1380, 1383, 1385, 1387–1388, 1390–1392, 1395–1397, 1399, 1401, 1403, 1409–1410, 1412, 1414, 1416, 1418–1419, 1421–1425, 1427–1430, 1432–1434, 1436–1442, 1446–1447, 1449–1454, 1456–1458, 1460–1461, 1463–1469, 1471, 1473–1475, 1477–1478, 1482, 1484–1485, 1487, 1491–1493, 1495, 1497–1499, 1501–1506, 1509–1511, 1514–1534, 1536–1537, 1539–1540, 1542–1559, 1562–1569, 1571–1572, 1574–1575, 1584–1586, 1588–1589, 1591–1595, 1598–1607, 1609–1616, 1618–1630, 1632–1635, 1637–1639, 1641–1644, 1647–1653, 1656–1662, 1665–1668, 1672, 1676–1677, 1679–1680, 1682, 1684–1685, 1687, 1690, 1705, 1707, 1709–1715, 1717, 1719–1727, 1729, 1737–1740, 1743–1750, 1752–1753, 1756–1757, 1759–1765, 1767–1768, 1770, 1784, 1786–1787, 1789–1790, 1792–1793, 1795–1798, 1800, 1803, 1808–1809, 1812–1814, 1816–1820, 1822–1824, 1826–1828, 1830–1832, 1834–1836, 1838–1839, 1841, 1845–1847, 1849–1855, 1858–1859, 1861, 1863–1866, 1868–1869, 1871–1872, 1874–1876, 1878–1879, 1882–1883, 1885, 1887–1888, 1890–1897, 1899, 1901–1904, 1906–1908, 1910–1914, 1916–1918, 1920–1922, 1924–1926, 1928–1932, 1935–1936, 1938–1941, 1943–1945, 1949–1950, 1952–1953, 1955–1958, 1962–1976, 1979–1982, 1984–1985, 1987–1990, 1993–1995, 1999, 2001–2024, 2027–2030, 2032, 2047–2050, 2052–2054, 2056, 2058, 2066–2067, 2069, 2071–2074, 2076–2079, 2081, 2087–2088, 2090, 2095–2096, 2098–2108, 2110–2121, 2125–2129, 2131, 2154, 2157–2158, 2161–2162, 2169–2170, 2176, 2178, 2198–2199, 2202–2203, 2206–2207, 2210–2211, 2214–2215, 2229–2231, 2233–2237, 2246, 2255–2256, 2259–2261, 2266–2267, 2270–2272, 2277–2278, 2281–2283, 2288–2289, 2291–2294, 2312–2313, 2345–2346, 2348, 2350–2351, 2354–2355, 2358–2359, 2362–2364, 2366, 2368, 2399, 2403–2404, 2406, 2411, 2417, 2419, 2424, 2429, 2433, 2440, 2450, 2453, 2457, 2461, 2465, 2469, 2474, 2478, 2482, 2487, 2492, 2497, 2501, 2506, 2510–2511, 2514, 2520, 2524, 2528, 2531, 2534, 2538, 2541, 2546, 2552, 2557, 2562, 2569, 2576, 2580, 2589–2590, 2595, 2600, 2605, 2609, 2611–2612, 2614–2615, 2617–2619, 2621, 2631, 2636, 2647–2656, 2674, 2676, 2683, 2685, 2687, 2689–2690, 2692–2693, 2695, 2697–2698, 2700–2701, 2703, 2705, 2710–2712, 2714, 2716, 2718–2722, 2724–2726, 2731, 2736–2738, 2740–2741, 2745–2747, 2749, 2751, 2753–2763, 2765, 2767, 2769–2779, 2781–2787, 2789, 2794, 2796, 2798–2799, 2802–2804, 2807–2810, 2815
sr.py	2024	2024	0%	8, 21, 23–42, 44, 46–49, 51–52, 54, 56, 58, 66, 68, 72–73, 75, 80–84, 86–88, 91, 103, 107, 111–119, 121, 123, 128–129, 131, 133–134, 136, 138, 140, 142–151, 153, 157, 159, 162–165, 167, 169–171, 176, 181, 184–186, 188–190, 192–193, 198–199, 202, 206–210, 212–213, 215–216, 221, 224–225, 228–234, 236–237, 239, 241, 244, 247, 249–257, 259, 262, 266–277, 279, 281–290, 293–294, 296, 298–300, 302, 304, 306–320, 322, 324–328, 334–337, 339–340, 344–346, 348–349, 351, 353–356, 358, 360, 362, 368, 370–371, 373–374, 376–377, 379, 382–383, 385–386, 388–391, 393, 396–407, 409, 413–418, 420–441, 443–446, 449, 452–455, 457–460, 462–464, 466, 468–475, 477–478, 481–484, 486, 488–489, 491–492, 494–496, 498–511, 514, 517, 519–521, 523, 525–526, 528–530, 532–534, 536–537, 539–540, 542–545, 547–550, 553–556, 558, 560–561, 563, 566–576, 578, 580–582, 584–585, 587, 589–593, 595, 597–602, 604–605, 607–613, 615, 617–620, 623, 627–640, 642, 644–649, 651, 653–654, 656–658, 661, 663–668, 674–681, 683, 685–696, 698–700, 702–704, 706, 708, 710–714, 716, 718, 723, 727–729, 731–732, 734, 736–737, 739–744, 746–747, 749, 754, 758–760, 762–763, 765, 767–768, 770–775, 777–778, 780, 785–786, 788–790, 792–794, 796, 798, 807, 809–810, 812–819, 823, 825, 828–830, 832, 834–835, 837–840, 842–845, 847, 849–851, 853, 855–856, 858, 860–861, 863, 866–867, 869–874, 876–877, 879–883, 885, 892–893, 895–896, 899–904, 906–909, 911–921, 923, 927–929, 931, 935, 937, 939–940, 942–955, 957, 961–962, 964–965, 967, 969–977, 979, 981–993, 996, 998–999, 1001–1006, 1008–1012, 1014–1016, 1018–1024, 1026–1027, 1031, 1038–1043, 1045–1047, 1049–1055, 1057–1064, 1066–1068, 1070–1073, 1075–1080, 1082–1084, 1086–1088, 1091–1093, 1095–1097, 1099–1100, 1102–1104, 1106–1113, 1118–1122, 1128–1131, 1133–1134, 1137–1139, 1141–1144, 1146, 1149, 1154–1156, 1158–1165, 1167–1168, 1170–1171, 1178–1180, 1182–1184, 1186, 1190, 1194, 1198–1201, 1203, 1206, 1208, 1210, 1212–1213, 1216–1218, 1220–1230, 1232–1234, 1237–1238, 1240–1249, 1252–1254, 1256–1261, 1263–1265, 1267–1269, 1271–1272, 1276–1282, 1284, 1286–1289, 1291–1295, 1299–1300, 1302–1303, 1305, 1308, 1310–1315, 1317–1325, 1327, 1329–1335, 1338–1352, 1356, 1362–1365, 1372–1373, 1376–1380, 1382–1386, 1389, 1396–1397, 1400–1403, 1405, 1407–1418, 1421–1424, 1426, 1428–1429, 1431–1439, 1441–1444, 1446–1450, 1452–1453, 1455–1457, 1459–1467, 1470, 1472–1476, 1478–1479, 1481–1482, 1484–1486, 1488–1489, 1492, 1494, 1496–1497, 1499–1500, 1502, 1504–1505, 1507, 1509–1512, 1514–1518, 1520–1525, 1527–1528, 1530, 1533–1536, 1538, 1540–1541, 1543–1551, 1553–1561, 1563–1564, 1566–1567, 1569, 1571–1572, 1574–1575, 1578, 1580–1584, 1586–1587, 1589–1590, 1592–1596, 1598–1600, 1602, 1604, 1608, 1610–1612, 1616–1621, 1623–1625, 1627, 1629, 1631, 1633, 1635, 1637–1639, 1641, 1643–1648, 1650, 1652, 1654, 1665–1668, 1670–1682, 1684–1687, 1696–1699, 1702–1705, 1707, 1709–1710, 1712–1720, 1723–1725, 1727–1728, 1730, 1732–1737, 1739–1740, 1742–1744, 1746–1748, 1750, 1752–1759, 1761, 1763, 1767–1768, 1771–1776, 1778, 1780–1784, 1786, 1788–1790, 1792, 1798–1799, 1802–1804, 1806–1816, 1818, 1820–1822, 1824, 1829–1831, 1833–1834, 1838–1840, 1844–1846, 1851, 1855–1857, 1861, 1863, 1865, 1869–1871, 1873–1874, 1876–1885, 1887–1888, 1890, 1892–1894, 1896–1898, 1901, 1903–1905, 1907–1908, 1910–1911, 1913–1914, 1916, 1918–1924, 1926–1927, 1929–1930, 1932–1934, 1936, 1942–1949, 1951–1953, 1955–1956, 1958, 1960, 1965–1968, 1972, 1977–1994, 1996–2001, 2004–2012, 2014–2017, 2019–2022, 2025–2026, 2028–2031, 2034–2037, 2039, 2041–2042, 2044–2052, 2054, 2060–2061, 2063–2067, 2069, 2072, 2074–2076, 2078–2083, 2085, 2087, 2093–2104, 2106, 2111, 2113–2115, 2117–2119, 2122, 2124–2130, 2132–2133, 2135, 2138–2140, 2142–2143, 2146–2149, 2151–2154, 2156–2159, 2161–2163, 2165–2167, 2169, 2172–2175, 2178, 2180–2184, 2186–2189, 2191–2194, 2196–2202, 2204, 2206–2210, 2212–2214, 2216–2226, 2228–2230, 2233–2234, 2236–2243, 2245–2247, 2249–2254, 2257–2262, 2264–2265, 2267–2269, 2271, 2275–2276, 2284–2285, 2287–2300, 2302–2314, 2316–2335, 2337–2343, 2345–2347, 2349–2350, 2352–2374, 2376, 2381, 2383–2385, 2387–2388, 2390, 2392–2393, 2395–2396, 2398, 2401–2402, 2404–2406, 2408–2409, 2411–2426, 2428–2434, 2436–2437, 2440–2445, 2447–2449, 2451–2457, 2459–2460, 2462–2468, 2470, 2472–2473, 2475–2477, 2479–2483, 2485–2489, 2491–2493, 2495–2496, 2498–2501, 2503–2506, 2508–2509, 2513–2514, 2516, 2518–2519, 2521, 2523, 2525–2530, 2532–2533, 2535, 2537, 2546–2547, 2561–2563, 2566–2567, 2570–2571, 2574, 2578–2580, 2582–2588, 2591, 2593, 2597, 2600–2601, 2603, 2605, 2612, 2619–2622, 2624–2631, 2633–2640, 2642–2643, 2645–2646, 2648–2650, 2652–2660, 2662–2667, 2669–2680, 2682–2685, 2687–2692, 2694–2705, 2707–2708, 2710–2711, 2713–2721, 2723–2732, 2734–2735, 2737–2739, 2741, 2743–2745, 2747–2748, 2750–2751, 2753–2756, 2758–2763, 2765–2768, 2770, 2773, 2778, 2781, 2784, 2787–2790, 2792, 2794, 2796–2797, 2799–2801, 2803, 2805–2807, 2809–2811, 2813–2814, 2816–2817, 2820, 2823, 2825–2826, 2829–2830, 2833, 2835, 2838, 2840–2843, 2846, 2848–2853, 2856, 2860–2863, 2865, 2867, 2880–2881, 2883–2886, 2888–2889, 2891, 2893, 2902–2903, 2905, 2915, 2919, 2924–2925, 2928, 2930–2932, 2934, 2936, 2941, 2951, 2956–2958, 2960, 2962–2963, 2966, 2972–2974, 2976–2978, 2980–2982, 2984–2987, 2989–2992, 2994–2995, 2997–2998, 3000–3001, 3003–3004, 3006–3007, 3009–3010, 3012–3013, 3015–3016, 3018–3019, 3021–3022, 3024–3028, 3030–3032, 3034–3035, 3037–3039, 3041–3043, 3045–3047, 3049–3051, 3053–3055, 3057
flow
__init__.py	1567	1491	4%	36, 122–123, 125, 127–129, 131–133, 140, 142–143, 148, 150–151, 156, 158–159, 161, 164–166, 169–174, 178, 181, 187–189, 191, 194, 197, 200–202, 205–206, 209, 211, 214–215, 219, 224–226, 228–229, 231, 235–236, 238–244, 246–253, 256–259, 261–262, 264, 266, 268, 270–272, 276–279, 281–285, 288–290, 292–296, 304–307, 309, 311–316, 318–321, 323–328, 334–337, 339–341, 348, 351–352, 354–364, 366, 368, 374–375, 378, 381–383, 388–391, 396–399, 401–405, 409–415, 417–421, 430–431, 433, 435, 437–443, 445–447, 449–450, 454–456, 458, 460–464, 466–467, 469, 471–474, 476, 478, 480–481, 483–484, 486–488, 490–491, 493, 496–498, 501–502, 504–506, 508–509, 511–512, 514, 516–517, 519, 521, 523, 525–526, 528, 531–532, 534, 536, 538, 540, 542, 546–547, 549–553, 557–559, 561, 564–565, 567–572, 574, 576–578, 580, 595, 597–600, 602–604, 606, 608–614, 616–624, 626, 628, 630–631, 633–638, 640, 642, 644–645, 647–664, 667–668, 671, 673–674, 705, 707–709, 711, 713, 716, 723–724, 730–732, 734–741, 747–749, 755, 757–760, 762, 768–770, 773–781, 784, 788, 790–791, 793–796, 801–803, 805–808, 810, 812–817, 819–821, 823–831, 833–842, 844–845, 847, 850, 852–853, 855–857, 862, 865, 867–872, 874–879, 883, 885–886, 888–889, 891–903, 905–908, 910, 912, 914–915, 917, 919, 921–928, 930–933, 935–938, 941, 944, 946, 950–951, 953–960, 965, 967–968, 970, 975, 977, 979, 981, 983, 985, 989–993, 997–1001, 1004–1007, 1009–1013, 1015–1017, 1020–1022, 1024, 1027–1028, 1030–1033, 1035, 1037–1041, 1047–1048, 1051–1052, 1054, 1058, 1061–1063, 1066–1067, 1069–1075, 1078–1081, 1083, 1085–1086, 1088, 1090–1096, 1098–1100, 1102–1103, 1106–1107, 1109, 1111, 1113, 1115–1117, 1123, 1127–1130, 1132–1137, 1139–1141, 1143–1151, 1153–1156, 1159–1163, 1165–1167, 1175–1180, 1182, 1184–1185, 1188–1192, 1194–1195, 1197, 1199–1202, 1204, 1206–1209, 1211–1213, 1215, 1217, 1222–1224, 1228, 1231, 1241, 1247, 1252, 1254, 1256–1260, 1262–1264, 1266, 1270–1273, 1275–1277, 1280, 1285–1292, 1294, 1297–1298, 1300–1301, 1305, 1323–1324, 1329–1330, 1336–1337, 1339, 1341, 1345–1347, 1349–1356, 1358–1359, 1361, 1363, 1368–1369, 1371, 1373–1374, 1376, 1378, 1380–1381, 1383–1386, 1389, 1391, 1394–1396, 1398, 1405, 1407–1416, 1418, 1424–1427, 1434, 1436, 1438–1444, 1447–1449, 1456–1458, 1460–1465, 1467–1469, 1471–1472, 1474, 1476–1477, 1479–1485, 1494, 1503–1509, 1511–1513, 1515–1516, 1519–1521, 1523–1524, 1526–1528, 1530, 1532, 1543–1546, 1548, 1550, 1554–1555, 1557–1558, 1560–1567, 1569–1570, 1572–1573, 1575–1580, 1584, 1588–1593, 1595–1598, 1600–1604, 1609–1610, 1612–1615, 1617–1621, 1626–1627, 1631–1634, 1636–1640, 1643–1644, 1646–1649, 1651–1655, 1658–1659, 1662, 1664–1677, 1680, 1683, 1686, 1688–1689, 1691–1692, 1696, 1698–1699, 1702–1704, 1707, 1710–1711, 1715–1717, 1719, 1721–1722, 1725–1731, 1735–1736, 1738–1739, 1741–1742, 1744–1747, 1749–1753, 1755, 1757, 1759–1760, 1762, 1766, 1776, 1778–1783, 1785, 1787, 1791, 1793–1798, 1800, 1802, 1804–1805, 1807, 1809–1810, 1812, 1815–1816, 1818–1819, 1821–1823, 1825–1833, 1835–1843, 1845, 1847–1849, 1851–1857, 1859–1860, 1862–1863, 1865–1868, 1870, 1872–1881, 1883–1884, 1886–1889, 1891, 1893–1897, 1899–1903, 1905–1907, 1917, 1919–1921, 1923–1926, 1928, 1930–1938, 1940–1941, 1943–1944, 1946, 1950, 1958–1961, 1963, 1965–1966, 1968–1969, 1972, 1976–1981, 1983, 1985, 1989, 1993–1995, 1999–2000, 2004, 2006–2007, 2009–2017, 2019–2021, 2025, 2029–2030, 2034–2035, 2037–2040, 2042, 2045–2048, 2051–2052, 2054–2055, 2057–2059, 2061–2062, 2064–2065, 2072–2073, 2075–2079, 2081–2082, 2084, 2086–2087, 2089–2090, 2092–2094, 2097–2104, 2106–2108, 2110–2113, 2117–2119, 2122–2128, 2130–2131, 2133–2134, 2137, 2140, 2142, 2144–2148, 2150–2153, 2155–2160, 2162, 2164–2165, 2167, 2169–2173, 2175, 2177–2181, 2183–2188, 2194, 2197, 2199, 2205–2206, 2208–2209, 2215, 2217–2218, 2224–2227, 2229–2230, 2232–2233, 2239–2245, 2247–2253, 2255–2265, 2267–2279, 2286–2308, 2315–2316, 2319–2320, 2322–2324, 2326, 2328–2331, 2333–2334, 2337–2339, 2343, 2347, 2351, 2353–2355, 2357–2358, 2360–2361, 2363–2367, 2370, 2372, 2374, 2376–2380, 2383–2388, 2390–2396, 2399, 2401, 2403–2408, 2411, 2413, 2415, 2417–2420, 2424–2425, 2428, 2432, 2434, 2437, 2439–2443, 2446–2450, 2452–2457, 2459, 2461, 2463, 2470, 2475–2477, 2479, 2481–2484, 2486–2487, 2489, 2491, 2493–2500, 2502–2503, 2505–2510, 2516–2527, 2529–2533, 2535–2544, 2553–2556, 2558, 2561–2564, 2566–2568, 2570–2572, 2574–2576, 2578–2580, 2588–2591, 2593–2596, 2598–2601
flowcb
log.py	181	181	0%	3–5, 7, 10, 26, 28–29, 31–40, 42–43, 45–47, 49, 51–58, 60–61, 63–65, 67, 69–71, 73–79, 81–82, 84–86, 88–94, 96–99, 101, 103–110, 112–114, 116–117, 119–121, 123–124, 126–129, 131, 133, 135–136, 138–144, 146, 148–150, 152–155, 159, 161, 163–168, 170–175, 177–183, 187, 189–191, 193–194, 196–203, 205–207, 209, 211–212, 214–215, 217–218, 220–225, 227–228, 230, 233, 237, 240–241, 244–245, 247–248, 250–253, 255–258, 260–264
flowcb/filter
wmo00_accumulate.py	122	122	0%	66–73, 75, 77, 80–87, 89–91, 94–98, 100–101, 104, 106–108, 110, 112, 114–119, 121–122, 124, 126, 128–130, 132, 135, 137–138, 140–143, 145–153, 155–157, 159–161, 164–171, 174–176, 178, 180, 184–187, 189, 191, 193, 196, 198–199, 201, 203–204, 206, 210, 214–216, 218–220, 222, 224–227, 229–230, 232–234, 236–237, 239–240, 244, 246–247, 249
wmo00_split.py	88	88	0%	46–53, 55, 57, 59–64, 66, 68–69, 71–75, 77–79, 81–84, 86–93, 95–98, 101–102, 104–105, 110–112, 114, 116, 118, 120–122, 124–128, 130–132, 134–135, 137–138, 140–141, 143, 145, 147–148, 150, 152, 155–156, 158–161, 163–167, 169
TOTAL	15611	12770	18%

Test Results

Tests	Skipped	Failures	Errors	Time
209	8 💤	0 ❌	0 🔥	16.959s ⏱️

[petersilva]

I went with wmo00_accumulate and wmo00_split ... which seem a bit more descriptive.
other suggestions. welcome.

[petersilva]

The other thing missing... is how will this be tested... no integration in flow_tests or unit tests...

[reidsunderland]

There's a typo here:

#        v
if 'baseURl' in msg:

[petersilva]

good eye!