Skip to content

Free online tool

Remove duplicate lines

Paste a list, or any text with one item per line, to remove the lines that repeat and keep the first of each in its original order. You can ignore differences in case or spacing, remove empty lines too, and see how many lines were removed. Nothing is uploaded.

Runs in your browser. Your text is never uploaded.

Runs in your browser. Your text isn’t uploaded or saved.

Options

How to remove duplicate lines

  1. 1Paste your list into the box, one item per line: email addresses, keywords, names or log lines.
  2. 2Choose whether “Apple” and “apple” count as the same, whether to ignore spaces at the ends of lines, and whether to remove empty lines.
  3. 3Check the result and the count of lines removed.
  4. 4Press Copy result and paste the clean list where you need it.

Features

  • Keeps the first copy of each line in the original order: nothing is sorted
  • Can ignore upper and lower case
  • Can ignore and trim spaces at the start and end of lines
  • Removes empty lines if you want, and says how many lines of each kind were removed
  • Lines that look the same match, even if their accents are stored differently
  • Free, no sign-up, and your text never leaves your browser

How duplicates are found

Here is exactly how lines are compared. Everything happens in your browser.

Lines
Windows (CR LF), old Mac (CR) and Unix (LF) line breaks are all recognised, and so are the line and paragraph separators (U+2028, U+2029) that InDesign, Word and some PDFs use, vertical tabs, form feeds and NEL. A line break at the very end of the text doesn’t start an extra, empty line. A line break at the end is kept, and the result uses Unix (LF) line breaks.
Duplicates
A line is a duplicate if the same line came earlier. The first copy stays where it is and later copies are removed, so the order never changes. Lines that look the same match even if an accent is stored differently (é as one character, or as e plus an accent).
Ignore case
Lines are compared without regard to upper and lower case, using Unicode case rules, so “Straße” and “STRASSE” match. The first copy keeps its own case.
Trim whitespace
Spaces and tabs at the start and end of each line are ignored when comparing, and removed from the lines that are kept.
Empty lines
Lines that are empty or hold only spaces are never treated as duplicates. Remove empty lines deletes all of them, and they’re counted separately.

Private by design

Your text stays with you

Everything runs in your browser. Nothing you type or paste is uploaded, logged or used for analytics, so it’s safe for code, contracts and private notes.

Works with any language

Accented letters, other scripts and emoji come through intact, and Windows, Mac and Unix line breaks are all understood.

Fast with long texts

Results update as you type. Long pastes are handled once you pause, so typing never stutters.

Frequently asked questions

Does it change the order of my lines?

No. The first time a line appears it stays where it is, and every later copy is removed, so the result is in the same order as your text. Nothing is sorted.

Which copy of a duplicate line is kept?

The first one. With “Ignore case” on, “Apple”, “apple” and “APPLE” count as the same line, and whichever comes first is kept exactly as you typed it.

What happens to blank lines?

Blank lines aren’t treated as duplicates of each other, because they usually separate groups of lines. They stay where they are, unless you turn on “Remove empty lines”, which removes every blank line, including lines of only spaces.

Can I remove duplicates from a spreadsheet column?

Yes. Copy the column and paste it here, and each cell becomes one line. Copy the result and paste it back into the spreadsheet, and each line goes back into a cell of its own.

Is my text uploaded or stored?

No. Everything happens in JavaScript in your browser, and your text is never sent to a server, logged or tracked. You can check by opening your browser’s network tab while you type. This tool doesn’t save your text either: it’s gone when you close or reload the page.

Is it free?

Yes. There’s no sign-up, paid plan or adverts, and no limit on how often you use it. Everything runs on your own device, so there are no servers to pay for.

More free text tools

Related tools

Looking for something else? Browse all free tools.