Suppose you have a text file and you need to remove all of its duplicate lines.TL;DRTo remove the duplicate lines while preserving their order in the file, use:read more

Source: LXer – How to remove duplicate lines from files with awk