Description: "rmdup" removes all the duplicate lines of a text file (or stdin), and output in the same order as input. Functions like sort+uniq, without changing the line order.
=====================
Motivation: When using zsh, one might wish not to record the commands which are already in history. Unfortunately, the zsh stores the command history instantly, therefore only the continuous duplicate lines would be discarded. With sort+uniq, the original order will be lost, and sometimes it crashes with encoding problem.
=====================
Algorithm: “rmdup” first (heap) sorts the lines (of a text file or from stdin) according to the text and delete/keep the duplicate lines like "uniq". Then it (heap) sorts the output according to the input order.
=====================
The source code is compressed with two precompiled binaries, the MacOS X 10.9 and Ubuntu 11 on Linux 3.0.0.14 (64 bit).
To Search:
File list (Check if you may need any files):
93317462rmdup.tar
Download