Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Remove & insertion of data in a same file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove & insertion of data in a same file
# 1  
Old 02-28-2007
Remove & insertion of data in a same file
I am iterating record by record through a file as below,

A,B
A,C
A,D
B,E
B,F
E,G
E,H

The same file should look like in the final output as below,

A,B
B,E
E,G
E,H
B,F
A,C
A,D

So while iterating, it should remove records from the same file & paste inside in the same file.

How can we achieve this functionality of removing & pasting records in the same file?

I can think of pseudo code as below;
CODE START
cat File1.txt | while read row; do

col1=`echo $row | awk -F","'{print $1}'`
col2=`echo $row | awk -F","'{print $2}'`

#if col2 has first column occurance in the file
awk /^"$col2"/ filename > Temp.txt

# HOW TO REMOVE CONTENTS from File1.txt which starts with $col2

# will echo Temp.txt >> File1.txt ... will paste the records in between?
done < File1.txt
CODE END

I hope I am clear in passing my problem.
# 2  
Old 02-28-2007
Code:
awk -F"," '
{ 
 print ; 
 str=$1 ;
 while ( getline )
 {
	while( str == $1 ) 
	{ str1 = str1 "\n" $0 ; getline } 
	txt = str1 txt
	str1=""
	print ;
	str=$1 ;
 } 
}
END { sub("^ *\n","",txt );print txt } ' < file > file

# 3  
Old 02-28-2007
This has deleted all the contents from the file. It is not working as expected.
# 4  
Old 02-28-2007
Try this
Code:
$ cat file
A,B
A,C
A,D
B,E
B,F
E,G
E,H
$ awk -F"," '
> {
> print > "file" ;
>  str=$1 ;
>  while ( getline )
>  {
>         while( str == $1 )
>         { str1 = str1 "\n" $0 ; getline }
>         txt = str1 txt
>         str1=""
>         if( $0 != "" ) print >"file";
>         str=$1 ;
>  }
> }
> END { sub("^ *\n","",txt );print txt > "file"} ' < file
$ cat file
A,B
B,E
E,G
E,H
B,F
A,C
A,D

# 5  
Old 02-28-2007
Hi Anbu, with hardcoded filename it works fine. How to make it work with filename indicated by a variable?


Say at the place of file if wish to use $fname?

Thanks for your co-operation. I tried using -v option but then I am getting some awk error.
# 6  
Old 02-28-2007
try this
Code:
awk -F"," -v file="$filenm" '
{
print > file  ;
 str=$1 ;
 while ( getline )
 {
        while( str == $1 )
        { str1 = str1 "\n" $0 ; getline }
        txt = str1 txt
        str1=""
        if( $0 != "" ) print > file;
        str=$1 ;
 }
}
END { sub("^ *\n","",txt );print txt > file} ' < $filenm

# 7  
Old 03-01-2007
Hi Anbu

But the same program could not work for the following combination -

A,B
A,C
A,D
B,F
B,E
E,G
E,H

For this output is appearing as -
A,B
B,F
E,G
E,H
B,E
A,C
A,D

Expected as -

A,B
B,F
B,E
E,G
E,H
A,C
A,D

If I have to say in text,
If you encounter some text in 2nd column & the same available as first column in the file those records should come after that in the same file.

I fought with the script you provided, but I couldnt succeed.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

To remove any invisible and special characters from the file(exclude @!#$&*)

Hi Guys, My requirement is to remove any invisible and special characters from the file like control M(carriage return) and alt numerics and it should not replace @#!$% abc|xyz|acd¥£ó adc|123| 12áí Please help on this. Thanks Rakesh (1 Reply)
Discussion started by: rakeshp
1 Replies

2. Shell Programming and Scripting

Remove data from file

Hi I have file with filename as below and need to remove based on date in it. Data in file: filename_092013.csv filename_082013.csv filename_072013.csv I want to remove filenames which are generated 15 days back. where date is generated in file. I need to create another file with... (8 Replies)
Discussion started by: cnrj
8 Replies

3. Shell Programming and Scripting

Sort a the file & refine data column & row format

cat file1.txt field1 "user1": field2:"data-cde" field3:"data-pqr" field4:"data-mno" field1 "user1": field2:"data-dcb" field3:"data-mxz" field4:"data-zul" field1 "user2": field2:"data-cqz" field3:"data-xoq" field4:"data-pos" Now i need to have the date like below. i have just... (7 Replies)
Discussion started by: ckaramsetty
7 Replies

4. UNIX for Dummies Questions & Answers

Remove part of file name with sed & mv

Ok, so I have bunch of files that are named "orange__file_name.asm" and I want to batch rename them to "file_name.asm" I know that using "ls | sed s/orange__//" will get rid of the part of the file name I do not want. But how do I combine that with the mv command to actually do it? Thanks JG (5 Replies)
Discussion started by: john galt
5 Replies

5. Shell Programming and Scripting

Remove 5th & 6th <td> from file

I have a page with a 6<td> structure. I need to strip the 5th and 6th <td></td> from the structure. The data in the 4th <td> varies, I can't get a pattern on it. So, I tried setting up a count of the td occurrence and then remove the 5th and 6th, but I'm not making any progress. Is there an easy,... (6 Replies)
Discussion started by: dba_frog
6 Replies

6. Shell Programming and Scripting

Remove 5th & 6th <td> from file

I have a page with a 6<td> structure. I need to strip the 5th and 6th <td></td> from the structure. The data in the 4th <td> varies, so I can't get a pattern on it. So, I tried setting up a count of the td occurrence and then remove the 5th and 6th, but I'm not making an progress. Is there an... (0 Replies)
Discussion started by: dba_frog
0 Replies

7. Shell Programming and Scripting

Extract data from an XML file & write into a CSV file

Hi All, I am having an XML tag like: <detail sim_ser_no_1="898407109001000090" imsi_1="452070001000090"> <security>ADM1=????</security> <security>PIN1=????</security> <security>PIN2=????</security> ... (2 Replies)
Discussion started by: ss_ss
2 Replies

8. Shell Programming and Scripting

Insertion in a file

Hi All, I want to insert a line just befor the lst line in a file. please can anyone advise. Cheers, Shazin (3 Replies)
Discussion started by: Shazin
3 Replies

9. UNIX for Dummies Questions & Answers

Search for & edit rows & columns in data file and pipe

Dear unix gurus, I have a data file with header information about a subject and also 3 columns of n rows of data on various items he owns. The data file looks something like this: adam peter blah blah blah blah blah blah car 01 30 200 02 31 400 03 57 121 .. .. .. .. .. .. n y... (8 Replies)
Discussion started by: tintin72
8 Replies

10. Shell Programming and Scripting

Remove Headers throughout a data file

I have a data file with over 500,000 records/lines that has the header throughout the file. SEQ_ID Name Start_Date Ins_date Add1 Add2 1 Harris 04/02/08 03/02/08 333 Main Suite 101 2 Smith 02/03/08 01/23/08 287 Jenkins SEQ_ID Name ... (3 Replies)
Discussion started by: psmall
3 Replies
Login or Register to Ask a Question