Shell Programming and Scripting

Try this:

Code:

dos2unix file.csv | sort > file-fixed-sorted.csv
uniq -c file-fixed-sorted.csv |  awk '$1==1{print $2}' > unique.csv
uniq -c file-fixed-sorted.csv |  awk '$1>1{print $2}' > duplicate.csv

@Corona688
That just turned my whole csv file into a huge block of text in 1 cell. However I think you are on to something! Some of my emails do seem to have a lot of trailing spaces at the end, so I need to figure out whats up with that. I tried something else to remove whitespaces and am now getting 140k results, so it's getting there.

I guess my original question has been answered at least, since it's least outputting SOMETHING now!

Then your 'csv' file has no newlines in it. Very odd, but perhaps fixable.

Code:

tr '\r' '\n' < inputfile > outputfile

Hey, that worked! 388k unique results. Thank you so much!!! I really appreciate yalls help. The duplicates one is still producing a blank results file, but now that my csv is fixed I was able to use a different chunk of code to produce duplicates.

This is going to make my life sooo much easier. Goodbye Excel!