missing ietf syslog translation - undefined syslog message juniper

[BlackkeeperRX]

Hello,

it seems like napalm-logs is missing some kind of translation for the ietf standard.

• Or an option to set syslog messages as ietf standard.
  I've configured the napalm-syslog server with basic configuration, no changes. Only set an ipaddress, port and disabled security.

Napalm-Logs is expecting the syslog message in following syntax:
<129>Oct 23 15:58:20 berlin cscript "message"

When i am looking at my tcpdump, juniper output as well as napalm log, the date format differs.

Juniper: Oct 23 15:58:20
TCP: 2017-10-23T16:02:38.950+02:00
napalm log: 2017-10-23T16:02:38.950+02:00

tcpdump:
Msg: 1 2017-10-23T16:02:38.950+02:00 berlin cscript - - - MX80 SN:XXXXX has booted 16.1R4-S4.3.
Uptime is 9 days, 23 hours, 7 minutes, 20 seconds

junos:
lab@berlin>show log messages | last 1
Oct 23 15:58:20 berlin cscript: MX80 SN:XXXXX has booted 16.1R4-S4.3. Uptime is 9 days, 23 hours, 3 minutes, 5 seconds

var/log/napalm/logs:
Dequeued message from <129>1 2017-10-23T16:02:38.950+02:00 berlin cscript - - - MX80 SN:XXXXX has booted 16.1R4-S4.3. Uptime is 9 days, 23 hours, 7 minutes, 20 seconds: 1508766848.76
2017-10-23 15:54:08,764,765 [napalm_logs.server][DEBUG ] Matching under junos
2017-10-23 15:54:08,765,765 [napalm_logs.server][DEBUG ] Matching using YAML-defined profiler:
2017-10-23 15:54:08,765,765 [napalm_logs.server][DEBUG ] <(\d+)>(\w+\s+\d+)\s+(\d\d:\d\d:\d\d)\s+(re\d.)?([^ ]+)\s+/?(\w+)[?(\d+)?]?:\s+([\w\s]+):(.*)
2017-10-23 15:54:08,765,765 [napalm_logs.server][DEBUG ] Match not found

Cheers!

[luke-orden]

Hi @BlackkeeperRX,

tcpdump looks at the packet that is sent to napalm-logs, so if that packet contains the time stamp 2017-10-23T16:02:38.950+02:00 then this is the info that napalm-logs will use. Is it possible that the message was not able to be sent to napalm-logs at the time that the first message was logged, so junos kept retying until it could send, resulting in different timestamps?

Do other log messages exhibit the behaviour?

[BlackkeeperRX]

Hi loverend,

this only occurs if structured messages in ietf syslog format will be send. Then napalm-logs will be unable to decode the format.
As long as i dont configure structured messages in the junos, the messages will be recognized.
But would be nice to have ietf standard available as parsing option

[luke-orden]

Hi,

Can you please provide the config required to enable ietf format so I can try to replicate.

Thanks

[BlackkeeperRX]

Hi loverend,

in junos, it is:
set system syslog host IPADDRESS structured-data

Best regards

[luke-orden]

To achieve this we will need to add a new prefix under https://github.com/napalm-automation/napalm-logs/blob/master/napalm_logs/config/junos/init.yml

[mirceaulinic]

Hi @BlackkeeperRX - I'm checking in here. Did Luke explanation help you to identify what's the prefix that needs to be added? When you have anything working, we'll welcome any contribution. Thanks! :-)