merging files and add missing rows


 
Thread Tools Search this Thread
Special Forums UNIX Desktop Questions & Answers merging files and add missing rows
# 1  
Old 09-13-2012
merging files and add missing rows

hello all,
I have files that have a specific way for naming the first column
they are make of five names in Pattern of 3
Y = [james mark john peter sam] (no case sensitive)
so the files are names $Y-$Y-$Y or $X-$Y-$Z depending how we look
they only exist of the pattern exist
now I want to create a file from them that if a specific patter for that column didnt exist it will write the pattern and write "NA" for the rest of the columns

so basically I should have 125 lines and if the match exist it will write itself
else print NA
Code:
for X in james mark john peter sam
do
	name=$(basename $X) 
	if [ -s $X ]
  	then
grep  count/$X-$X-$X file01  | awk ‘{print $0}’ >> file02.txt
else
echo  "count/$X-$X-$X NA NA NA NA" >> file02.txt
fi
done

I am doing this but its not working not sure what I am doing wrong SmilieSmilieSmilie
can you please help
# 2  
Old 09-13-2012
Show your input files and desired output. Your above description looks lit bit Smilie...
# 3  
Old 09-13-2012
I have this now which print the 125 right but all with NA Smilie and dont get the pattern right

Code:
for X in james mark john peter sam 
do
for Y in james mark john peter sam
do
for Z in james mark john peter sam 
do
name=$(basename $X) 
	if [ -s $X ]
  	then
grep  -o count/$X-$Y-$Z file01  | awk ‘{print $0}’ >> file02.txt
else
echo  "count/$X-$Y-$Z NA NA NA NA" >> file02.txt
fi
done 
done 
done

# 4  
Old 09-13-2012
Please check below..

Quote:
Originally Posted by A-V
I have this now which print the 125 right but all with NA Smilie and dont get the pattern right

Code:
for X in james mark john peter sam 
do
for Y in james mark john peter sam
do
for Z in james mark john peter sam 
do
name=$(basename $X) # Why you need this - As X is james mark john peter sam,
    if [ -s $X ] # And What you are checking here.....? X is only names...
      then
grep  -o count/$X-$Y-$Z file01  | awk ‘{print $0}’ >> file02.txt # what you are searching here?.. For "count/$X-$Y-$Z" pattern .??
else
echo  "count/$X-$Y-$Z NA NA NA NA" >> file02.txt
fi
done 
done 
done



Assuming you are searching for pattern "$X-$Y-$Z"

Code:
for X in james mark john peter sam 
do
for Y in james mark john peter sam
do
for Z in james mark john peter sam 
do
if [[ `grep "$X-$Y-$Z" -c file01` -gt 0 ]]
then
grep  -o "$X-$Y-$Z" file01  >> file02.txt  
else
echo  "$X-$Y-$Z NA NA NA NA" >> file02.txt
fi
done 
done 
done


Last edited by pamu; 09-13-2012 at 09:37 AM..
# 5  
Old 09-13-2012
silly mistake by me for grep
my file names in first column look like this
count/james-james-mark
count/james-mark-mark
so basically count/$X-$Y-$Z
for example one of my files has 60 lines with values looking like this
count/james-james-mark3.5 2.25 2.5 2.25 2.33333 3.55556 3.33333 2.88889
count/peter-peter-peter 3.33333 4.22222 1 0 2 0 4 0
sp 60 out of 125 can be covers with these lines and the rest should be the pattern and NAs

Last edited by A-V; 09-13-2012 at 10:44 AM..
# 6  
Old 09-13-2012
Quote:
Originally Posted by A-V
silly mistake by me for grep
my file names in first column look like this
count/james-james-mark
count/james-mark-mark
so basically count/$X-$Y-$Z
Have you tried this..

Code:
for X in james mark john peter sam 
do
for Y in james mark john peter sam
do
for Z in james mark john peter sam 
do
if [[ `grep "$X-$Y-$Z" -c file01` -gt 0 ]]
then
grep  -o "count/$X-$Y-$Z" file01  >> file02.txt  
else
echo  "$X-$Y-$Z NA NA NA NA" >> file02.txt
fi
done 
done 
done

This User Gave Thanks to pamu For This Post:
# 7  
Old 09-13-2012
yap, if the line exist it only prints
count/$X-$Y-$Z and not the rest of the row
I can see now that GREP -O is not correct for start
solved I guess ... it should be -w instead of -o to get the whole line Smilie

but dont know y I dont get 125 for all the files in one directory

Last edited by A-V; 09-13-2012 at 11:04 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Merging rows based on same ID in First column.

Hellow, I have a tab-delimited file with 3 columns : BINPACKER.13259.1.p2 SSF48239 BINPACKER.13259.1.p2 PF13243 BINPACKER.13259.1.p2 G3DSA:1.50.10.20 BINPACKER.13259.2.p2 SSF48239 BINPACKER.13259.2.p2 PF13243 BINPACKER.13259.2.p2 G3DSA:1.50.10.20... (7 Replies)
Discussion started by: anjaliANJALI
7 Replies

2. Shell Programming and Scripting

Merging rows after matching a pattern

Hi All, I have the below file where I want the lines to merged based on a pattern. AFTER CMMILAOJ CMMILAAJ AFTER CMDROPEJ CMMIMVIJ CMMIRNTJ CMMIRNRJ CMMIRNWJ CMMIRNAJ CMMIRNDJ AFTER CMMIRNTJ CMMIRNRJ CMMIRNWJ (4 Replies)
Discussion started by: varun22486
4 Replies

3. Shell Programming and Scripting

Merging rows in awk

Hello, I have a data format as follows: Ind1 0 1 2 Ind1 0 2 1 Ind2 1 1 0 Ind2 2 2 0 I want to use AWK to have this output: Ind1 00 12 21 Ind2 12 12 00 That is to merge each two rows with the same row names. Thank you very much in advance for your help. (8 Replies)
Discussion started by: Homa
8 Replies

4. Shell Programming and Scripting

Merging rows using two common rows.

Hi.. My requirement is simple but unable to get that.. File 1 : 3 415 A G 4 421 G . 39 421 G A 2 421 G A,C 41 427 A . 4 427 A C 42 436 G . 3 436 G C 43 445 C . 2 445 C T 41 447 A . Output (4 Replies)
Discussion started by: empyrean
4 Replies

5. UNIX for Dummies Questions & Answers

merging rows into new file based on rows and first column

I have 2 files, file01= 7 columns, row unknown (but few) file02= 7 columns, row unknown (but many) now I want to create an output with the first field that is shared in both of them and then subtract the results from the rest of the fields and print there e.g. file 01 James|0|50|25|10|50|30... (1 Reply)
Discussion started by: A-V
1 Replies

6. UNIX for Dummies Questions & Answers

Merging two text files by a column and filling in the missing values

Hi, I have to text files that I want to merge by the first column. The values in the first column pretty much match for the first part. However there are some values that are present in column 1 and not present in column 2 or vice versa. For such values I would like to substitute X for the... (9 Replies)
Discussion started by: evelibertine
9 Replies

7. Shell Programming and Scripting

Merging rows with same column 1 value

I have the following space-delimited input: 1 11.785710 117.857100 1 15 150 1 20 200 1 25 250 3 2.142855 21.428550 3 25 250 22 1.071435 10.714350 The first field is the ID number, the second field is the percentage of the total points that the person has and the third column is the number... (3 Replies)
Discussion started by: mdlloyd7
3 Replies

8. Shell Programming and Scripting

csv to xls : missing rows

A unix script generates a file "1.csv". I use the following to email this as an excel sheet. /usr/bin/uuencode /tmp/1.csv 1.csv > $PATH/attachment.txt mailx -r abc@domain.com -s "Subject" myself@domain.com < $PATH/attachment.txtI get the file as CSV in the email and everything is fine except... (9 Replies)
Discussion started by: girish1428
9 Replies

9. Shell Programming and Scripting

add new rows in list of files

Hi, I Have a directory(views) and i have 100 .sql files in the same directory. I need to be add 8 rows in each file. The format 8 rows has like: grant select on file_name to User1 / grant select on file_name to User2 / grant select on file_name to User3 / grant select on file_name to... (1 Reply)
Discussion started by: koti_rama
1 Replies

10. Shell Programming and Scripting

Merging of rows

Hi guys, Wish you all a very Happy New Year!!!. Thanks in advance. I want to read a file and merge the rows which have '\n' in it. The rows could be > 50,000 bytes. The script should merge all the rows till the next row starts with word 'Type|'. ex.... (24 Replies)
Discussion started by: ssachins
24 Replies
Login or Register to Ask a Question