Fast and flexible log normalization library
Fast and flexible log normalization library
We have just released liblognorm 2.0.4. This new version mainly provides new parser support options, like different JSON number formats and unix timestamps. See the Changelog for details. Version 2.0.4, 2017-10-04 added support for native JSON number formats supported by parsers: number, float, hex added support for creating unix timestamps supported by parsers: date-rfc3164, date-rfc5424 […]
We have just released liblognorm 2.0.3. This new version provides some fixes for the the annotate function and adds a test for it. A few different issues have also been fixed. See the Changelog for details. Changelog:Version 2.0.3, 2017-03-22 add ability to load rulebase from a string introduces new API: int ln_loadSamplesFromString(ln_ctx ctx, const char […]
A rulesbase is a collection of rules. The rulebase is the core of the normalizing process as it holds the information that is needed to transform logs into a common format. Upon execution of the normalizer, it will be transferred into a parse-tree.
A rule is a a scheme that fits to a specific type of logfile from a specific device. It consists of multiple fields, which reflect the type of information. Many of these rules together build the rulebase.
A first example for a rulebase you can download at http://blog.gerhards.net/2010/11/log-normalization-first-results.html I will use an excerpt of that rulebase to show you the most common expressions. rule=:%date:date-rfc3164% %host:word% %tag:char-to:\x3a%: no longer listening on %ip:ipv4%#%port:number%’ That excerpt is a common rule. A rule contains different “parts”/properties, like the message you want to normalize (e.g. Host, IP, […]