Author: Adiscon Support

  • liblognorm 1.0.0 released

    We have just released liblognorm 1.0.0. This is a completely revamped and enhanced version. It introduces incompatible API changes, which were unavoidable. For details please visit

    http://www.liblognorm.com/news/on-liblognorm-1-0-0/

    Changes

    Version 1.0.0, 2013-11-28

    • WARNING: this version has incompatible interface and older programs will not compile with it.
      For details see http://www.liblognorm.com/news/on-liblognorm-1-0-0/
    • libestr is not used any more in interface functions. Traditional C strings are used instead. Internally, libestr is still used, but scheduled for removal.
    • libee is not used any more. JSON-C is used for object handling instead. Parsers and formatters are now part of liblognorm.
    • added new field type “rest”, which simply sinks all up to end of the string.
    • added support for glueing two fields together, without literal between them. It allows for constructs like:
      %volume:number%%unit:word%
      which matches string “1000Kbps”
    • Fix incorrect merging of trees with empty literal at end
      Thanks to Pavel Levshin for the patch
    • this version has survived many bugfixes

     

    Download:
    http://www.liblognorm.com/download/liblognorm-1-0-0/

    As always, feedback is appreciated.

    Best regards,
    Florian Riedl

  • liblognorm 1.0.0

    Download file name: liblognorm 1.0.0

    liblognorm 1.0.0
    sha256 hash: 01f73c735fd7291c2f1041518f4d005a8100e6be33f07eab00d4f27d1c1934de

    Author: Rainer Gerhards (rgerhards@adiscon.com)
    Version: 1.0.0 File size: 0.349 MB

    Download this file now!

  • On liblognorm 1.0.0

    Liblognorm is a fast-samples based normalization library. It’s brand new version 1.0.0 will be released today. It is a major improvement over previous versions, but unfortunately we needed to change the API. So some notes are due.

    Liblognorm evolves since several years and was intially meant to be used primarily with the Mitre CEE effort. Consequently, the initial version of liblognorm (0.x) uses the libee CEE support library in its API.

    As time evolved, the initial CEE schema underwent considerable change. Even worse, Mitre lost funding for CEE. While the CEE ideas survived as part of Red Hat-driven “Project Lumberjack”, the data structures became greatly simplified and JSON based. That effectively made libee obsolete (and also in parts libestr, which was specifically written to support CEE’s initial requirement of embedded NUL chars in strings).

    Recently, Pavel Levshin converted liblognorm to native JSON, which helped improve performance and simplicity for many client applications. Unfortunately, this change broke interface compatibility (and there was no way to avoid that, obviously…).

    The current library is the result of that effort. Application developers are encouraged to switch to this version, as it provides the benefit of a simpler API. This version is now being tracked by the git master branch.

    However, if you need to stick to the old API, there is a git branch liblognorm0, which contains the previous version of the library. This branch is also maintained for important bug fixes, so it is safe to use.

    We recommend that packagers create packages both for liblognorm0 and liblognorm1. Note that liblognorm’s development packages cannot coexist on the same system as the PKGCONFIG system would get into trouble. Adiscon’s own packages follow this schema.

    Note that rsyslog will soon begin to enjoy the benefits of liblognorm1. This resulst in a notable performance improvement for mmnormalize. Support will initially become available in v8.

  • liblognorm 0.3.7 released

    We have just released liblognorm 0.3.7. This release contains a single API extension which is needed by John Hopper’s python bindings (and also contributed by him – thanks!)

    Changes

    Version 0.3.7, 2013-07-17

    • added support to load single samples
      Thanks to John Hopper for the patch

    Download:
    http://www.liblognorm.com/download/liblognorm-0-3-7/

    As always, feedback is appreciated.

    Best regards,
    Florian Riedl

  • liblognorm 0.3.7

    Download file name: liblognorm 0.3.7

    liblognorm 0.3.7
    sha256 hash: b000ab7aab9ed3d7a2c23303463b5bcceb7bb6538714bd41248d02913a82e223

    Author: Rainer Gerhards (rgerhards@adiscon.com)
    Version: 0.3.7 File size: 0.339 MB

    Download this file now!

  • liblognorm 0.3.6 released

    We have just released liblognorm 0.3.6.

    Changes

    Version 0.3.6, 2013-03-22

    •   ugfix: unitialized variable could lead to rulebase load error

    Download:
    http://www.liblognorm.com/download/liblognorm-0-3-6/

    As always, feedback is appreciated.

    Best regards,
    Florian Riedl

  • liblognorm 0.3.6

    Download file name: liblognorm 0.3.6

    liblognorm 0.3.6
    sha256 hash: e2cf27027905c7be91c891b5ac9304a88a5c1981cd36011c84b45de29336b111

    Author: Rainer Gerhards (rgerhards@adiscon.com)
    Version: 0.3.6 File size: 0.315 MB

    Download this file now!

  • Log Normalization and special characters

    When trying to normalize log messages via liblognorm and mmnormalize, you need to create a rulebase first. The rulebase is usually a representation of message formats.

    Due to the format of these rules, it is necessary to be cautious. Some messages and rule necessities could possibly cause confusion to the configuration interpreter. This mainly applies to clear text passages in single rules.

    For example, if you have a log message from a Cisco ASA, the message looks like this:

    2012-11-23T10:47:42+01:00 10.10.10.10 : %ASA-3-313001: ...

    The only interesting parts are the IP and the numerical code to identify the message. We are not interested in the timestamp or “%ASA”. But when making the rule, the trouble starts there. The percent character is also used to define variables and their values in a rule. Thus it needs to be escaped. This is done with the ASCII code representation of the percent character. The rule would look like this:

    rule=: %date:word% %host:ipv4% : x25ASA-%char1:char-to:-%-%char2:number%: ...

    If you write “%ASA” into the rule, the interpreter will think, that a new variable starts there. This will cause confusion to the rest of the rule and render it not working correctly. This needs to be avoided.

    The same applies to “:”. But this time, it needs to be escaped when using it as delimiter vor variables. Example:

    %variable:char-to:x3a%

    This will fill “variable” with everything until the next “:” occurs. If you just put a “:” here as a delimiter, the rule will not work anymore.

  • liblognorm 0.3.5 released

    We have just released liblognorm 0.3.5.

    Changes

    Version 0.3.5 (rgerhards), 2012-09-18

    • renamed “normalizer” tool to “lognormalizer” to solve name clashes
      Thanks to the Fedora folks for pointing this out.:

     

    Download:
    http://www.liblognorm.com/download/liblognorm-0-3-5/

    As always, feedback is appreciated.

    Best regards,
    Florian Riedl

  • liblognorm 0.3.5

    Download file name: liblognorm 0.3.5

    liblognorm 0.3.5
    sha256 hash: 5714f145b379adb64fe1d87ea6adec9e8d91e5fe1bc0654febb85960dc34dd06

    Author: Rainer Gerhards (rgerhards@adiscon.com)
    Version: 0.3.5 File size: 0.315 MB

    Download this file now!