Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

file preprocessing

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting file preprocessing
# 1  
Old 12-23-2008
file preprocessing
hi experts,

I have a file let's say FileA.

FileA
====
2 a b c
2 g h i
3 l k m
3 o p a
4 r t f
........
.........

I wanna an output file which contains as below :

Outputfile:

2 g h i
3 o p a
4 r t f
.........
.........

I mean to say it should compare the 1st column values and if duplicate values found then it will delete the 1st record...

Plz help me.

Thanks in advance.

-subhendu-
# 2  
Old 12-23-2008
Code:
#! /usr/bin/perl -w
open FH,"<a.txt";
while(<FH>){
	@tmp=split(" ",$_);
	$hash{$tmp[0]}=$_;
}
for $key (keys %hash){
	print $hash{$key};
}

# 3  
Old 12-23-2008
Hi,

try:

Command:
Code:
sed -n '/^[0-9]\+/{N;s/\([0-9]\+\).*\n\(\1.*\)/\2/;p}' file9

Input:
Code:
FileA
====
2 a b c
2 g h i
3 l k m
3 o p a
4 r t f

Output:
Code:
2 g h i
3 o p a

HTH Chris
# 4  
Old 12-23-2008
thnx for reply...

but as i mentioned i wanna the output as

Outputfile:

2 g h i
3 o p a
4 r t f
………
………

my requirement:
A shell script in unix(AIX).

thanks,
Subhendu
# 5  
Old 12-23-2008
try this

cat file
2 a b c
2 g h i
3 l k m
3 o p a
4 r t f

awk '{print $1}' file | sort -u | while read line; do grep ^$line file | tail -1 ; done
2 g h i
3 o p a
4 r t f

this will works even the first feild is not a digit
Last edited by reddyrajal; 12-23-2008 at 04:49 AM..
# 6  
Old 12-23-2008
Sorry paste the wrong snipped. It's nearly the same:

Code:
sed '/^[0-9]\+/{N;s/\([0-9]\+\).*\n\(\1.*\)/\2/};/^[0-9]/!d' file9

or as sedfile

Code:
sed -f sedfile file9

sedfile:
Code:
/^[0-9]\+/{
N
s/\([0-9]\+\).*\n\(\1.*\)/\2/}
/^[0-9]/!d

Login or Register to Ask a Question

Previous Thread | Next Thread

4 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script (sh file) logic to compare contents of one file with another file and output to file

Shell script logic Hi I have 2 input files like with file 1 content as (file1) "BRGTEST-242" a.txt "BRGTEST-240" a.txt "BRGTEST-219" e.txt File 2 contents as fle(2) "BRGTEST-244" a.txt "BRGTEST-244" b.txt "BRGTEST-231" c.txt "BRGTEST-231" d.txt "BRGTEST-221" e.txt I want to get... (22 Replies)
Discussion started by: pottic
22 Replies

2. Shell Programming and Scripting

Compare 2 text file with 1 column in each file and write mismatch data to 3rd file

Hi, I need to compare 2 text files with around 60000 rows and 1 column. I need to compare these and write the mismatch data to 3rd file. File1 - file2 = file3 wc -l file1.txt 58112 wc -l file2.txt 55260 head -5 file1.txt 101214200123 101214700300 101250030067 101214100500... (10 Replies)
Discussion started by: Divya Nochiyil
10 Replies

3. Shell Programming and Scripting

Match list of strings in File A and compare with File B, C and write to a output file in CSV format

Hi Friends, I'm a great fan of this forum... it has helped me tone my skills in shell scripting. I have a challenge here, which I'm sure you guys would help me in achieving... File A has a list of job ids and I need to compare this with the File B (*.log) and File C (extend *.log) and copy... (6 Replies)
Discussion started by: asnandhakumar
6 Replies

4. Shell Programming and Scripting

AWK and preprocessing

Hello , I'm trying to to write simple ANSI C preprocessor but I have problem with recursive call. When I call my function : preproc(filename) , I get message " Segmentation fault Core Dumped : " . Please Help Paweł Łepko Hire Is My Code : #!/usr/bin/nawk -f #Autor :... (4 Replies)
Discussion started by: scotty_123
4 Replies
Login or Register to Ask a Question