mergelog

mergelog is a small and fast C program, which merges HTTP log files by date in 'Common Log Format' (Apache default log format) from Web servers, behind round-robin DNS. It has been designed to easily process huge logs from highly stressed servers, and can manage gzipped files.

Tags Internet Log Analysis Logging Web Site Management
Licenses GPL
Operating Systems POSIX Linux Unix BSD FreeBSD
Implementation C

Tweet this project Short link

Rss Recent releases

  • Rrelease-mid
  •  11 Apr 2001 09:44
  • Rrelease-after

Changes: The corrupted log lines problem which stopped the mergelog process has been solved.

  • Rrelease-mid
  •  21 Mar 2001 12:35
  • Rrelease-after

Changes: A major bug which could break the date sorting of the first log line entry of each log file has been fixed.

  • Rrelease-mid
  •  05 Mar 2001 06:45
  • Rrelease-after

Changes: This release fixes a potential segmentation fault with bad formatted log lines, and packaging has been slightly improved.

  • Rrelease-mid
  •  31 Jan 2001 09:34
  • Rrelease-after

Changes: The project has switched to switched to the autoconf standard.

  • Rrelease-mid
  •  30 Jan 2001 06:13
  • Rrelease-after

    Changes: Fixes for all known bugs, and optimization of the new date comparison system introduced in 4.0.

    Rss Recent comments

    Rcomment-before 13 Jan 2006 19:30 Rcomment-trans roger_byucougar Rcomment-after

    Re: no dots in ip

    > Works well -- seems to have a small bug

    > if the hostname/ipaddress does not

    > contain a period [calls it an error but

    > still outputs it, so not a big

    > problem].

    Seems that the problem is not that hostname's lack a period, but rather if they are of size 3 or less then mergelog mis-parses them. To fix change SCAN_OFFSET to 7.

    Rcomment-before 05 Jan 2006 13:04 Rcomment-trans roger_byucougar Rcomment-after

    no dots in ip
    Works well -- seems to have a small bug if the hostname/ipaddress does not contain a period [calls it an error but still outputs it, so not a big problem].

    Thanks.

    -Roger

    Rcomment-before 03 Aug 2004 16:45 Rcomment-trans smdas Rcomment-after

    Very handy!
    mergelog should come with apache. The compiled binary is under 40k (I know, I know, platform, toolchain, etc... but still!).

    How about a mergelog for error_log's? These are trickier because many parsers don't output in apache error log format, but chunks of contiguous aberrant errors could be treated as a single line, and it's date/time could be flagged "after" the first properly formatted log line above it, but before any possible mergeable line. Perhaps an option to strip any malformed (consider corrupted?) lines.

    Rcomment-before 29 Jun 2002 08:56 Rcomment-trans cybert Rcomment-after

    Re: Damn fast

    > Ok, I'm new to this whole thing. I
    > downloaded it and compiled it. I have 3
    > log files on 3 different servers. nce I
    > pull them down to a local machine, I
    > want to merge them. How do I do this
    > using mergelog?

    You can try the following command:
    mergelog logfile1 logfile2 logfile2 > merge

    You can also try this:
    man mergelog

    ;-)

    Rcomment-before 26 Jun 2002 13:45 Rcomment-trans jrthor2 Rcomment-after

    Re: Damn fast
    Ok, I'm new to this whole thing. I downloaded it and compiled it. I have 3 log files on 3 different servers. nce I pull them down to a local machine, I want to merge them. How do I do this using mergelog?

    > Ok. I`m perl zealot, but this time i
    > should admit C speed against perl
    > analogue merge_apache_logs.
    > It saved me. I`ve successfully merged 3
    > log files with total of 1.5GB like in 2
    > minutes. Webalizer parsed output without
    > problems.
    > Its simple, fast and it does what it
    > should.

    Ab16b8957f23fa7801b84b909ef263db_thumb

    Project Spotlight

    NOC

    A Network Operation Center's Operation Support System.

    9537a730c2f9da79cb353a5463ac3237_thumb

    Project Spotlight

    AQEMU

    A QEMU and KVM GUI.