[HELP] - Delete rows on a CSV file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting [HELP] - Delete rows on a CSV file
# 1  
Old 06-19-2009
Error [HELP] - Delete rows on a CSV file

Hello to all members,

I am very new in unix stuff (shell scripting), but a want to learn a lot. I am a ex windows user but now i am absolutely Linux super user... Smilie

So i am tryng to made a function to do this:

I have two csv files only with numbers, the first one a have:

1
2
3
4
5

in the second file i have:

2
3
5

So the result i want in my new file is:

1
4


I compare the two files and all the numbers that are not equal it will be written in a new file.

Until now i can only grep the numbers that are the same in both files.

for number in `cat <File1.csv>; do
grep $number <File2.csv>
done

Can anyone help me with this?

Thanks
# 2  
Old 06-20-2009
There are several ways to do this, but this follows the example you gave. You have to use the file with the fewer lines to grep the file with more. This isn't 100% accurate though. Suppose you have 6 in file2 and not in file1, then it would not be printed. You can create two loops and reverse the files on the second one to catch this.

Code:
for num in `cat File2.csv`
do 
    strng=${strng:-"($num"}${strng:+"|$num"} 
done

strng=${strng}${strng:+")"}  # creates '(2|3|5)'

egrep -v $strng File1.csv

You could also do this with a simple diff, but then you have to format the output properly.

Code:
diff num1.txt num2.txt
1d0
< 1
4d2
< 4

# 3  
Old 06-20-2009
Quote:
Originally Posted by Sadarrab
I have two csv files only with numbers, the first one a have:

These are not CSV files.
Quote:

1
2
3
4
5

in the second file i have:

2
3
5

So the result i want in my new file is:

1
4


I compare the two files and all the numbers that are not equal it will be written in a new file.

Until now i can only grep the numbers that are the same in both files.

When you post code, please wrap it in [code] tags.
Quote:
Code:
for number in `cat <File1.csv>; do
 grep $number <File2.csv>
done


This prints the numbers that are in file1 but not in file2:

Code:
grep -vf file2  file1

# 4  
Old 06-20-2009
Code:
# awk 'FNR==NR{a[$0];next}!($0 in a)' file2 file1
1
4

# 5  
Old 06-21-2009
Java

Quote:
Originally Posted by ghostdog74
Code:
# awk 'FNR==NR{a[$0];next}!($0 in a)' file2 file1
1
4

So i made this...

# awk 'FNR==NR{a[$0];next}!($0 in a)' file2 file1 > file3

I get all the numbers that are not repeaded between file2 and file1 and i write them in a new file.
But i didn't urderstand what this means "'FNR==NR...".

Thanks a lot.
# 6  
Old 06-21-2009
read the manual
# 7  
Old 06-22-2009
Quote:
Originally Posted by Sadarrab
...
I compare the two files and all the numbers that are not equal it will be written in a new file.

Until now i can only grep the numbers that are the same in both files.
...
Assuming that the numbers are sorted in each file, the following command prints numbers from either file that are not common:

Code:
join -a1 -a2 -o1.1,2.1 file1 file2 | awk 'NF==1'

tyler_durden
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Get duplicate rows from a csv file

How can i get the duplicates rows from a file using unix, for example i have data like a,1 b,2 c,3 d,4 a,1 c,3 e,5 i want output to be like a,1 c,3 (4 Replies)
Discussion started by: ggupta
4 Replies

2. Shell Programming and Scripting

Bash script help - removing certain rows from .csv file

Hello Everyone, I am trying to find a way to take a .csv file with 7 columns and a ton of rows (over 600,000) and remove the entire row if the cell in forth column is blank. Just to give you a little background on why I am doing this (just in case there is an easier way), I am pulling... (3 Replies)
Discussion started by: MrTuxor
3 Replies

3. Shell Programming and Scripting

Converting rows to columns in csv file

Hi, I have a requirement to convert rows into columns. data looks like: c1,c2,c3,.. r1,r2,r3,.. p1,p2,p3,.. and so on.. output shud be like this: c1,r1,p1,.. c2,r2,p2,.. c3,r3,p3,.. Thanks in advance, (12 Replies)
Discussion started by: Divya1987
12 Replies

4. Shell Programming and Scripting

Copying down first row in to all the below blank rows in a .csv file

Hi All, I have many of files(.csv) of the format given below. Date,Name,Location 04/02/2012,A,India ,B,China ,C,USA Like this I have 1000's of rows and many columns in all my files. I need a shell script to copy down the Date(in this example column1) to the next 2 rows below(in the... (8 Replies)
Discussion started by: ks_reddy
8 Replies

5. Shell Programming and Scripting

Parsing a CSV file and deleting all rows on condition

Hello list, I am working on a csv file which contains two fields per record which contain IP addresses. What I am trying to do is find records which have identical fields(IP addresses) which occur 4(four) times, and if they do, delete all records with that specific identical field(ip address). ... (4 Replies)
Discussion started by: landossa
4 Replies

6. Shell Programming and Scripting

How to remove Blank rows in a csv file

Hi, I need help to remove blank rows at the end of file. Sample data: "Oslo, Symra kino",Oslo,130-7,Symra 1,130-7-91 "Tønsberg, Brygga Kino SF",Tønsberg,202-1,Tønsberg SF 4,202-1-4 ,,,, ,,,, ,,,, ,,,, Expected data: "Oslo, Symra kino",Oslo,130-7,Symra 1,130-7-91 "Tønsberg, Brygga... (6 Replies)
Discussion started by: cnraja
6 Replies

7. Shell Programming and Scripting

Please do help: Perl Script to pull out rows from a CSV file

I have CSV file that contains data in the format as shown below: ABC, 67, 56, 67, 78, 89, 76, 55 PDR, 85, 83, 83, 72, 82, 89, 83 MPG, 86, 53, 54, 65, 23, 54, 75 .. .. .. .. I want to create a script that will pull out the rows from the above sheet and paste it into another CSV file.... (12 Replies)
Discussion started by: pankajusc
12 Replies

8. Shell Programming and Scripting

delete rows in a file based on the rows of another file

I need to delete rows based on the number of lines in a different file, I have a piece of code with me working but when I merge with my C application, it doesnt work. sed '1,'\"`wc -l < /tmp/fileyyyy`\"'d' /tmp/fileA > /tmp/filexxxx Can anyone give me an alternate solution for the above (2 Replies)
Discussion started by: Muthuraj K
2 Replies

9. Shell Programming and Scripting

Deleting rows from csv file

Hello, I am supposed to process about 100 csv files. But these files have some extra lines at the bottom of the file. these extra lines start with a header for each column and then some values below. These lines are actually a summary of the actual data and not supposed to be processed. These... (8 Replies)
Discussion started by: cobroraj
8 Replies

10. Shell Programming and Scripting

How to delete particular rows from a file

Hi I have a file having 1000 rows. Now I would like to remove 10 rows from it. Plz give me the script. Eg: input file like 4 1 4500.0 1 5 1 1.0 30 6 1 1.0 4500 7 1 4.0 730 7 2 500000.0 730 8 1 785460.0 45 8 7 94255.0 30 9 1 31800.0 30 9 4 36000.0 30 10 1 15000.0 30... (5 Replies)
Discussion started by: suresh3566
5 Replies
Login or Register to Ask a Question