D
danielb
Guest
We are currently in the process of implementing an ELK stack for log file archival, searching, etc. and the hard(est) thing we had to do was create the GROK filters for Logstash. The format of the log files across the AppServer broker, Server, NameServer and Database are all slightly different. I'm posting them here, in hopes that they will be useful to other developers that need to achieve the same thing, as well as to get some comments/feedback on them from anyone else that has implemented the same. OpenEdge Database: ^%{OE_TIMESTAMP:[@metadata][timestamp]}\s*%{OE_PROCESS_ID
id}\s*%{OE_THREAD_ID:thread_id}\s*%{WORD:message_type}\s+%{WORD:db_process_type}\s*(%{OE_USER_ID:user_id})?\:\s+\(%{OE_MESSAGE_NUMrogress_message_num}\)\s+%{GREEDYDATA:message_text} OpenEdge AppServer/Webspeed: ^%{OE_TIMESTAMP:[@metadata][timestamp]}\s*%{OE_PROCESS_IDid}\s*%{OE_THREAD_ID:thread_id}\s*%{BASE10NUM:logging_level}\s+%{NOTSPACE:exec_env}\s*%{NOTSPACE:exec_subsys}\s*%{GREEDYDATA:message_text}\s*\(%{BASE10NUMrogress_message_num}\)$ or ^%{OE_TIMESTAMP:[@metadata][timestamp]}\s*%{OE_PROCESS_IDid}\s*%{OE_THREAD_ID:thread_id}\s*%{BASE10NUM:logging_level}\s+%{NOTSPACE:exec_env}\s*%{NOTSPACE:exec_subsys}\s*%{GREEDYDATA:message_text} We have also defined the following types specifically for the patterns: OE_PROCESS_ID \P\-[0-9]* OE_THREAD_ID \T\-[a-zA-Z0-9\-]* OE_TIMESTAMP %{SYSLOG5424SD} OE_USER_ID %{BASE16FLOAT} OE_DB_PROCESS_TYPE %{WORD} OE_DB_MESSAGE_TYPE [FI] OE_MESSAGE_NUM \(?%{BASE16FLOAT}\)? Hopefully someone else finds this information useful.
Continue reading...
id}\s*%{OE_THREAD_ID:thread_id}\s*%{WORD:message_type}\s+%{WORD:db_process_type}\s*(%{OE_USER_ID:user_id})?\:\s+\(%{OE_MESSAGE_NUMrogress_message_num}\)\s+%{GREEDYDATA:message_text} OpenEdge AppServer/Webspeed: ^%{OE_TIMESTAMP:[@metadata][timestamp]}\s*%{OE_PROCESS_IDid}\s*%{OE_THREAD_ID:thread_id}\s*%{BASE10NUM:logging_level}\s+%{NOTSPACE:exec_env}\s*%{NOTSPACE:exec_subsys}\s*%{GREEDYDATA:message_text}\s*\(%{BASE10NUMrogress_message_num}\)$ or ^%{OE_TIMESTAMP:[@metadata][timestamp]}\s*%{OE_PROCESS_IDid}\s*%{OE_THREAD_ID:thread_id}\s*%{BASE10NUM:logging_level}\s+%{NOTSPACE:exec_env}\s*%{NOTSPACE:exec_subsys}\s*%{GREEDYDATA:message_text} We have also defined the following types specifically for the patterns: OE_PROCESS_ID \P\-[0-9]* OE_THREAD_ID \T\-[a-zA-Z0-9\-]* OE_TIMESTAMP %{SYSLOG5424SD} OE_USER_ID %{BASE16FLOAT} OE_DB_PROCESS_TYPE %{WORD} OE_DB_MESSAGE_TYPE [FI] OE_MESSAGE_NUM \(?%{BASE16FLOAT}\)? Hopefully someone else finds this information useful.Continue reading...