mergelog
mergelog is a small and fast C program, which merges HTTP log files by date in 'Common Log Format' (Apache default log format) from Web servers, behind round-robin DNS. It has been designed to easily process huge logs from highly stressed servers, and can manage gzipped files.
| Tags | Internet Log Analysis Logging Web Site Management |
|---|---|
| Licenses | GPL |
| Operating Systems | POSIX Linux Unix BSD FreeBSD |
| Implementation | C |
Recent releases
- 11 Apr 2001 09:44
Changes: The corrupted log lines problem which stopped the mergelog process has been solved.
- 21 Mar 2001 12:35
Changes: A major bug which could break the date sorting of the first log line entry of each log file has been fixed.
- 05 Mar 2001 06:45
Changes: This release fixes a potential segmentation fault with bad formatted log lines, and packaging has been slightly improved.
- 31 Jan 2001 09:34
Changes: The project has switched to switched to the autoconf standard.
- 30 Jan 2001 06:13
Changes: Fixes for all known bugs, and optimization of the new date comparison system introduced in 4.0.
- All comments
Recent commentsRe: no dots in ip
> Works well -- seems to have a small bug
> if the hostname/ipaddress does not
> contain a period [calls it an error but
> still outputs it, so not a big
> problem].
Seems that the problem is not that hostname's lack a period, but rather if they are of size 3 or less then mergelog mis-parses them. To fix change SCAN_OFFSET to 7.
no dots in ip
Works well -- seems to have a small bug if the hostname/ipaddress does not contain a period [calls it an error but still outputs it, so not a big problem].
Thanks.
-Roger
Very handy!
mergelog should come with apache. The compiled binary is under 40k (I know, I know, platform, toolchain, etc... but still!).
How about a mergelog for error_log's? These are trickier because many parsers don't output in apache error log format, but chunks of contiguous aberrant errors could be treated as a single line, and it's date/time could be flagged "after" the first properly formatted log line above it, but before any possible mergeable line. Perhaps an option to strip any malformed (consider corrupted?) lines.
Re: Damn fast
> Ok, I'm new to this whole thing. I
> downloaded it and compiled it. I have 3
> log files on 3 different servers. nce I
> pull them down to a local machine, I
> want to merge them. How do I do this
> using mergelog?
You can try the following command:
mergelog logfile1 logfile2 logfile2 > merge
You can also try this:
man mergelog
;-)
Re: Damn fast
Ok, I'm new to this whole thing. I downloaded it and compiled it. I have 3 log files on 3 different servers. nce I pull them down to a local machine, I want to merge them. How do I do this using mergelog?
> Ok. I`m perl zealot, but this time i
> should admit C speed against perl
> analogue merge_apache_logs.
> It saved me. I`ve successfully merged 3
> log files with total of 1.5GB like in 2
> minutes. Webalizer parsed output without
> problems.
> Its simple, fast and it does what it
> should.