Multiple file handling


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Multiple file handling
# 1  
Old 04-26-2007
Multiple file handling

Dear All,

I have two files, which looks like:

File 1

124
235
152
178
156
142
178
163
159

File 2
124|5623
452|6698
178|9995
235|7542
159|8852
152|9963
156|8512
885|9956
754|6523


I have to search all values of File1 in File2 and make an output file which should look like:

124|5623
235|7542
152|9963
178|9995
156|8512
142|
178|9995
163|
159|8852

The sequence of output file Should be in accordance to File 1. If any Value of File 1 is not found in File 2 then second field
of Output file should be a blank.
I cannot use grep command in for loop as the number of entries in File 1 is about 5000 and that in File 2 is 900,000.


Regards
Rochit
# 2  
Old 04-26-2007
just one of many ways:
Code:
awk 'BEGIN {  
        while ( (getline line < "file2")  > 0 ) {
              n = split(line,a,"|")
              array[a[1]] = a[2]
        }
    } 
    {  print $1 "|" array[$1]  }
' "file1"

output:
Code:
# ./test.sh
124|5623
235|7542
152|9963
178|9995
156|8512
142|
178|9995
163|
159|8852

# 3  
Old 04-26-2007
Thanks but it is taking lot of time to execute and I am afraid I cannot devote that much of time. So If u could please suggest another solution which gives an output faster than this one.
And Please could u explain how the code u have given works.

Thanks & regards
Rochit
# 4  
Old 04-26-2007
my bad. i missed the 900000 records. well how about trying something else. if you have Python, this is an alternative:
Code:
data=open("file1").readlines() #read file1 into array
data=[i.strip() for i in data] #get rid of newlines
for line in open("file2"):     #process file2 line by line
    if line[0:3] in data: #check only first 3 characters
        print line,
    else: print "%s|" % line[0:3]

# 5  
Old 04-28-2007
hi,
sorry to say but I think ,y server does has Python installed in it. I can execute awk and perl scripts.
Thanks & regards
Rochit
# 6  
Old 04-28-2007
sorry, i don't code perl, but roughly the method is same.
not tested code:
Code:
open(file1,"<","file1") or die "cannot open file1:$!";
#get file1 into array
while ( <file1> ) {
   push @file1, chomp($_); 
}
open(file2,"<","file2") or die "cannot open file2:$!";
#go through every line in file2
while ( my $line = <file2> ) {
  chomp($line);
  # get the first 3 characters, check if its in file1 array, if yes print
  if ( exists $file1[substr($line,0,3)] ) {
   print $line;
  }
}
close(file1);
close(file2);

# 7  
Old 04-28-2007
Code:
awk -F"|" ' NR == FNR { arr[$1]=$2; next } {  print $1"|"arr[$1] } ' file2 file1

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Using getopts for handling multiple options

Hi Guys, I have created a script for our automated DB creation, it works fine with default option(-d). $ ./test_db.ksh -d abc 11 dev -d is Default option ORACLE_SID=abc ORACLE_VERSION=11 ENV_TYPE=dev For creating a customized DB, i thought of giving the user different options.... (8 Replies)
Discussion started by: veeresh_15
8 Replies

2. Shell Programming and Scripting

ISSUE in handling multiple same name files :-(

Dear all, My work is completely stuck cos of the following issue. Please find it here and kindly help me. Task is following: I have set of files with such pattern 1t-rw-rw-r-- 1 emily emily 119 Jun 11 10:45 vgtree_5_1_pfs.root 3t-rw-rw-r-- 1 emily emily 145 Jun 11 10:46 vgtree_5_3_pfs.root... (4 Replies)
Discussion started by: emily
4 Replies

3. Shell Programming and Scripting

Help with Handling multiple argument in shell script

Hi i have written a shell script that takes only single ip address from the user and calculates its latency and reliability, can you please tell me that what should be done if i want that user should enter 100 or 1000 ip address (5 Replies)
Discussion started by: Preeti_17
5 Replies

4. Shell Programming and Scripting

[SOLVED] Handling multiple files using awk

Hi, I am trying to process 2 files simultaneously using awk satisfying following condition, Both files contain 3 columns. It should take entry from column 1 from first file, look for that entry in file 2 and if found, add column 2 and column 3 from both files and output to third file. For e.g.... (4 Replies)
Discussion started by: muazfarooqaslam
4 Replies

5. UNIX for Advanced & Expert Users

please help me in file handling

sir i have to get first line from a file for example >cat file1 abc zxc asd adsf from that file1 i need only first line expected result >abc please help me ! (1 Reply)
Discussion started by: ponmuthu
1 Replies

6. Shell Programming and Scripting

handling multiple files using awk command and wants to get separate out file for each

hai all I am new to the world of shell scripting I wanted to extract two columns from multiple files say around 25 files and i wanted to get the separate outfile for each input file tired using the following command to extract two columns from 25 files awk... (2 Replies)
Discussion started by: hema dhevi
2 Replies

7. Programming

Handling Multiple terminals

Hi, Basically I've written a game in ncurses that supports multiple players. Each player has a process associated with him which shares a segment of memory in which the player's structures are stored, and these structured are accessed by the 'server' program and handled there. The scope of the... (13 Replies)
Discussion started by: dgre0018
13 Replies

8. Shell Programming and Scripting

Handling multiple fields of a database file for toupper() function in awk

hello everyone.... script is: To convert the contents of a database file into uppercase my code is: printf "%s\n" , $2 | awk '{print toupper($2)}' emp.lst i m able to do only for one field.....didn't get any sources for handling multiple fields. please suggest me for multiple... (1 Reply)
Discussion started by: Priyanka Bhati
1 Replies

9. Shell Programming and Scripting

Gen. Question - Script calls multiple programs - Return Code Handling?

General Question: If a script calls multiple external programs (external to the script, but still on unix), where do the return codes go? Let's say one of external programs fails, does the entire script fail and send a non-zero return code to the job scheduling software, or is the return code sent... (1 Reply)
Discussion started by: jnanasakti
1 Replies

10. Programming

File Handling in C

Hi all, I have a problem in handling files through C. here is the problem im having: i will query the database (for instance consider employees table ) for empno,ename,job,salary fields.The query returns me some 100 of rows. now i need to place them in a file in row wise pattern as they... (3 Replies)
Discussion started by: trinath
3 Replies
Login or Register to Ask a Question