Jan 27, 2009

De-Dupe Lists With An Excel Formula

MailChimp automatically de-dupes your email lists whenever you import them. We’ll even spit out the rejected emails, in case you want to see who the dupes were.

But some people like to do things manually, because they enjoy pain.

So here’s an Excel formula that you can use to compare and remove duplicates from two lists…

Instructions for removing duplicates from your list with Excel:

  1. Open up Microsoft Excel.
  2. Paste your entire list into Column A.
  3. Sort the entire column alphabetically (A > Z). This is very important.
  4. Click on and highlight Column B.
  1. With Column B highlighted (press the gray B button above the column to highlight the entire column), enter the following into the "formula" box in Excel.
=IF(TRIM(A1)=TRIM(A2),"",TRIM(A1))
  1. After the formula is filled in, and with the entire Column B still selected, go to the menu in Excel, and choose: Edit > Fill > Down.
  2. This will fill Column B with a new, updated list, with all duplicates removed.
  3. Now, you’ll want to copy that column into a new spreadsheet. "Copy & Paste" won’t work…
  4. Select Column B, and choose Edit > Copy.
  5. Create a new spreadsheet, and select Edit > Paste Special > "Values".
  6. Now you should have the final, updated list. You can sort it now, and remove blank lines.

Is the page BLANK? After you paste into a new sheet, blank entries from your list show up at the top, so if you have a very large list, you might have a lot of blank lines near the top. If so, scroll down to see your list.