You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
the popularity scripts that parse classAds were crashing yesterday due to corrupt timestamps. This is perhaps a "new" issue as @vkuznet did not have the same problem when he last ran the popularity scripts.
Most are just extra 000 or 000000 at the end (but there is one as bad as '1517880783000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000').
David, I'm not sure I can do anything about it in CMSSpark. The corrupted fields comes from data and not from the code. And, how data appears on HDFS is also out of my scope. We can keep this issue as a reminder or close it, but I don't really see if I can change anything in CMSSpark code itself to address this issue.
the popularity scripts that parse classAds were crashing yesterday due to corrupt timestamps. This is perhaps a "new" issue as @vkuznet did not have the same problem when he last ran the popularity scripts.
Most are just extra 000 or 000000 at the end (but there is one as bad as '1517880783000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000').
An example of one not easily patched is
/HIMinBiasUPC/HIRun2011-v1/RAW,null,0,production,RECOHID11,-3859345924948754432,8.058,100.81333333333335,20180205,0.07992990345192434,RAW
these are isolated to a few dates
dataset-20180204.csv
dataset-20180205.csv
(so perhaps I am making this issue against the wrong repo)
anyway, I protected the popularity script for parsing classads against this problem, but I thought I would report it.
The text was updated successfully, but these errors were encountered: