So in my file called file_name, i want to extract only the values MSISDN,NAM,OBO,TS,etc... append the results in the OUTPUT file then use the paste command to put them in the same line.
The script is working fine for a smaller size of file. But with a file size of 5GB it's 2 days running.
Please i need help!
Last edited by Corona688; 02-15-2017 at 01:38 PM..
I'm not surprised it takes days to run, you are running shell externals 22 times per line, and processing the entire file each time when you probably only meant to process a line. So you are processing the file 22*n times more than you needed to, with n being the number of lines in the file.
How about:
Code:
awk '{ A[$1]++ }
END {
for(X in A) { printf("%s%s", P, X); P="\t" }
printf("\n");
P="" ;
for(X in A) { printf("%s%s", P, A[X]); P="\t" }
printf("\n"); }' < inputfile > outputfile
If that doesn't work, please show the input you have and the output you want.
Without reasonable, representative input and output samples it's difficult to see WHAT you really want done. To me it seems you have count records in the file, and want to print the records' corresponding lines, blockwise. (Which is not what you specify verbally: "... extract only the values MSISDN,NAM,OBO,TS,etc...", except $1 is the only field in a line)
- Are a single record's lines contiguous? Or do records overlap?
- Are the input records in the order that you want them printed?
- Are there more fields in a line or just $1? Should those be printed?
- Are there more lines that you want suppressed?
Let's say this is the input, 2(actually they are 4milion) blocks of line.
(dn:serv=...) begining of a block and space the end of block.
In each block, i want to extract only the MSISDN,IMSI,NAM,TS11,21,22,BAIC,APNID,OBO,OBI then for each block the expected values should be in the same line separated by semi colon or colon.
Hi everyone,
I have a csv file which has data with different heading and column names as below.
Static Data Ingested
,,,,,,,,,,,,Known Explained Rejections
Column_1,column_2,Column_3,Column_4,,Column_6,Column_7,,% Column_8,,Column_9 ,Column_10 ,... (14 Replies)
Hi Guys,
I am new to shell script.I need your help to write a shell script.
I need to write a shell script to extract data from a .csv file where columns are ',' separated.
The file has 7 columns having values say column 1,column 2.....column 7 as below along with their values.
Name, Address,... (7 Replies)
Hi Guys,
I am new to shell script.I need your help to write a shell script.
I need to write a shell script to extract data from a .csv file where columns are ',' separated.
The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (1 Reply)
hi
i am trying to extract some specific data out of a text file using regular expressions with shell script
that is using a multiline grep .. and the tool i am using is pcregrep so that i can get compatibility with perl's regular expressions
for a sample data like this, i am trying to grab... (6 Replies)
If I have a log like :
Mon Jul 19 05:07:34 2010; TCP; eth3; 52 bytes; from abc to def
Mon Jul 19 05:07:35 2010; UDP; eth3; 46 bytes; from aaa to bbb
Mon Jul 19 05:07:35 2010; TCP; eth3; 52 bytes; from def to ghi
I will need an output like this :
Time abc to def... (1 Reply)
Hello Scripting Gurus,
I need help with extracting data from the XML file using shell script.
The data is in a large XML and I need to extract the id values of all completedworkflows. Here is a sample of it. Input and output data is also in the attached text files.
<wfregistry>... (5 Replies)
give me a shell-script which extract data from log file on a server by giving date and time as input (for both start time and end time) and it will give the logs generated during the given time as output. (4 Replies)
Hi,
This is the xml file that i have.
- <front-servlet platform="WAS4.0" request-retriever="SiteMinder-aware" configuration-rescan-interval="60000">
<concurrency-throttle maximum-concurrency="50" redirect-page="/jsp/defaulterror.jsp" />
- <loggers>
<instrumentation... (5 Replies)
Hello All,
I am starting with UNIX. Any help is highly appreciated.
How to extract data using UNIX shell script? And how do you export data using UNIX shell scripts into Microsoft Excel format?
Thank you. (3 Replies)
02-15-2017
