Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

awk scrpting

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users awk scrpting
# 1  
Old 06-02-2009
awk scrpting
Hi

I have 2 files of large size( 10 Miilions) , i want to join 2 files based on some condition .
for this taking lot of time and 100 cpu .i want to iterate the based on some value (like 1 lakh)
I put the 2 files in the associative arrays . if the array limit reaches the 1 lakh join the with the second file . this process is repeate at last line of the first file .

i want to take 1 lakh elements into array . read the full second file if matches prinrt to some file ,after that store another 1 lakh elements into array read the second file matches it store in to same file of above. it process is like finish the first file .
# 2  
Old 06-02-2009
No duplicate or cross-posting, please read the rules.

Continue here:

https://www.unix.com/shell-programmin...scripting.html

Thread closed.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk output yields error: awk:can't open job_name (Autosys)

Good evening, Im newbie at unix specially with awk From an scheduler program called Autosys i want to extract some data reading an inputfile that comprises jobs names, then formating the output to columns for example 1. This is the inputfile: $ more MapaRep.txt ds_extra_nikira_usuarios... (18 Replies)
Discussion started by: alexcol
18 Replies

2. Shell Programming and Scripting

Passing awk variable argument to a script which is being called inside awk

consider the script below sh /opt/hqe/hqapi1-client-5.0.0/bin/hqapi.sh alert list --host=localhost --port=7443 --user=hqadmin --password=hqadmin --secure=true >/tmp/alerts.xml awk -F'' '{for(i=1;i<=NF;i++){ if($i=="Alert id") { if(id!="") if(dt!=""){ cmd="sh someScript.sh... (2 Replies)
Discussion started by: vivek d r
2 Replies

3. Shell Programming and Scripting

HELP with AWK one-liner. Need to employ an If condition inside AWK to check for array variable ?

Hello experts, I'm stuck with this script for three days now. Here's what i need. I need to split a large delimited (,) file into 2 files based on the value present in the last field. Samp: Something.csv bca,adc,asdf,123,12C bca,adc,asdf,123,13C def,adc,asdf,123,12A I need this split... (6 Replies)
Discussion started by: shell_boy23
6 Replies

4. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

5. Shell Programming and Scripting

Problem with awk awk: program limit exceeded: sprintf buffer size=1020

Hi I have many problems with a script. I have a script that formats a text file but always prints the same error when i try to execute it The code is that: { if (NF==17){ print $0 }else{ fields=NF; all=$0; while... (2 Replies)
Discussion started by: fate
2 Replies

6. Shell Programming and Scripting

Explain How ":=" works in shell scrpting

Hi , can any one explain how in the below code ":=" working in shell programming. L_DEBUG=${L_DBG:=N} if ; then set -x fi Thanks muddasas (2 Replies)
Discussion started by: muddasani
2 Replies

7. Shell Programming and Scripting

awk: assign variable with -v didn't work in awk filter

I want to filter 2nd column = 2 using awk $ cat t 1 2 2 4 $ VAR=2 #variable worked in print $ cat t | awk -v ID=$VAR ' { print ID}' 2 2 # but variable didn't work in awk filter $ cat t | awk -v ID=$VAR '$2~/ID/ { print $0}' (2 Replies)
Discussion started by: honglus
2 Replies

8. Shell Programming and Scripting

scripting/awk help : awk sum output is not comming in regular format. Pls advise.

Hi Experts, I am adding a column of numbers with awk , however not getting correct output: # awk '{sum+=$1} END {print sum}' datafile 2.15291e+06 How can I getthe output like : 2152910 Thank you.. # awk '{sum+=$1} END {print sum}' datafile 2.15079e+06 (3 Replies)
Discussion started by: rveri
3 Replies

9. Shell Programming and Scripting

Awk problem: How to express the single quote(') by using awk print function

Actually I got a list of file end with *.txt I want to use the same command apply to all the *.txt Thus I try to find out the fastest way to write those same command in a script and then want to let them run automatics. For example: I got the file below: file1.txt file2.txt file3.txt... (4 Replies)
Discussion started by: patrick87
4 Replies

10. Shell Programming and Scripting

URGENT!! few unix/scrpting/ksh qns

hey guys, I have a few general qns, so I'd appreciate quick responses. 1. How do u read a file line by line and display its contents using KSH? 2. How to kill a process using only the name? 3. How to create an empty file with 1MB memory size? 4. How to append some content to an existing... (3 Replies)
Discussion started by: sasuke_uchiha
3 Replies
Login or Register to Ask a Question