A first example for a rulebase you can download at
http://blog.gerhards.net/2010/11/log-normalization-first-results.html
I will use an excerpt of that rulebase to show you the most common expressions.
rule=:%date:date-rfc3164% %host:word% %tag:char-to:\x3a%: no longer listening on %ip:ipv4%#%port:number%'
That excerpt is a common rule. A rule contains different “parts”/properties, like the message you want to normalize (e.g. Host, IP, Source, Syslogtag…)
All rules have to start with “rule=:”
The buildup of a property is as follows
%field name:field type:additional information%
field name -> that name can be free selected. It should reflect the content of the field, e.g. src-ip for the source IP. In common sense, the field names should be the same in all samples, if the content of the field means the same.
field type -> selects the accordant parser
date-rfc3164: date in format of rfc3164
ipv4: ip adress
number: sequence of numbers (example: %port:number%)
word: everything till the next blank (example: %host:word%)
char-to: the field will be defined by the sign in the additional information (example: %tag:char-to:\x3a%: (x3a means ":" in the additional information))
additional information -> dependent on the field type; some field types need additional information
In our example we have some more information that is used as “simple text”. That parts are exactly like the parts in the messages and are not selected by a property.
Very important:
In the field type “char-to” you can use any item that is on your keyboard. In the case shown above, the item “:” has to be escaped with it’s ANSII version. Other characters do not have to be escaped.