Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Field Number read and "wc -l"

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Field Number read and "wc -l"
# 1  
Old 05-27-2013
Field Number read and "wc -l"
Hallo Friends,

I have +/- 800K files that i need to go through. The files have extention .csv
The script that i need should read field number 14 and if field number 14 is equal to 6010 or 2345 or 5690 or 8670 or 4567 then i need to know the total count.

Code:
1110,1,1367366253,51805a5300b856be,1367366227,1367366228,1367366229,9992,,0873636707,873536707,0861243662,861243662,9101,187,1,,1367366227.613,1367366227.668,1367366228.487,1367366228.507,1367366229.297,1367366229.297,1367366253.828,1367366253.828,1367366253.888,1367366253.848,10,,2,,2,3,8090,,,,,,,,,,MGC-CDR-NODE-STRING,24.531,26.275,,d4aa5f00b2ba11e296be0003babf4db9,,10.0.102.13,196.23.114.146,BW015707572010513988490743@196.35.130.5,10.0.102.13,10.0.4.35,196.23.114.225,G729,G729,20548,17986,,0,,,,,,,0,

---------- Post updated at 03:17 PM ---------- Previous update was at 02:49 PM ----------

in other words i would like the total number of occurences where filed 14 is equal to (6010 or 2345 or 5690 or 8670 or 4567 ) in all the 800K files.
Last edited by Scrutinizer; 05-29-2013 at 04:12 PM.. Reason: code tags instead of icode tags
# 2  
Old 05-27-2013
Code:
awk -F"," ' $14 ~ /6010|2345|5690|8670|4567/ { print $0 ; ++cpt } END { print cpt} ' < file

# 3  
Old 05-27-2013
Fundix i am getting an error
Code:
awk -F"," ' $14 ~ /6010|2345|5690|8670|4567/ { print $0 ; ++cpt } END { print cpt} ' < *.csv

-bash: *.csv: ambiguous redirect

---------- Post updated at 04:18 PM ---------- Previous update was at 04:04 PM ----------

i tried

cat *_201304*.csv|awk -F"," ' $14 ~ /9100/9101/9102/9103/9104/9105/9106/9150/9151/9152/9153/9200/9201/9202/9203/9300/9301/9302/9303/9304/9305/9400/9401/9510/9511/ { print $0 ; ++cpt } END { print cpt} '

it bring back

Code:
 cat  *_201304*.csv|awk -F"," ' $14 ~ /9100/9101/9102/9103/9104/9105/9106/9150/9151/9152/9153/9200/9201/9202/9203/9300/9301/9302/9303/9304/9305/9400/9401/9510/9511/ { print $0 ; ++cpt } END { print cpt} '
awk:  $14 ~ /9100/9101/9102/9103/9104/9105/9106/9150/9151/9152/9153/9200/9201/9202/9203/9300/9301/9302/9303/9304/9305/9400/9401/9510/9511/ { print $0 ; ++cpt } END { print cpt}
awk:                                                                                                                                       ^ syntax error

Last edited by Scrutinizer; 05-29-2013 at 04:13 PM.. Reason: code tags
# 4  
Old 05-27-2013
Delete the < before *.csv
# 5  
Old 05-27-2013
i removed it and i get a syntax error.
# 6  
Old 05-27-2013
Im not good at awk, you can try to loop through the files such that the awk command takes one file at a time.
else try this,

Code:
cat *.csv | awk -F"," ' $14 ~ /6010|2345|5690|8670|4567/ { print $0 ; ++cpt } END { print cpt} '

PS: cat is not a good idea in your case since you say its +/- 800K files.
# 7  
Old 05-27-2013
Hallo Arun

i tried i get a syntax error

{ print $0 ; ++cpt } END { print cpt} == are we supposed to have spaces there?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Failure: if grep "$Var" "$line" inside while read line loop

Hi everybody, I am new at Unix/Bourne shell scripting and with my youngest experiences, I will not become very old with it :o My code: #!/bin/sh set -e set -u export IFS= optl="Optl" LOCSTORCLI="/opt/lsi/storcli/storcli" ($LOCSTORCLI /c0 /vall show | grep RAID | cut -d " "... (5 Replies)
Discussion started by: Subsonic66
5 Replies

2. UNIX for Dummies Questions & Answers

Using "mailx" command to read "to" and "cc" email addreses from input file

How to use "mailx" command to do e-mail reading the input file containing email address, where column 1 has name and column 2 containing “To” e-mail address and column 3 contains “cc” e-mail address to include with same email. Sample input file, email.txt Below is an sample code where... (2 Replies)
Discussion started by: asjaiswal
2 Replies

3. Shell Programming and Scripting

Awk,sed : change every 2nd field ":" to "|"

Hi Experts, I have a string with colon delimited, want 2nd colon to be changed to a pipe. data: 101:8:43:4:72:14:41:69:85:3:137:4:3:0:4:0:9:3:0:3:12:3: I am trying with sed, but can change only 1 occurance: echo "101:8:43:4:72:14:41:69:85:3:137:4:3:0:4:0:9:3:0:3:12:3:" | sed 's/:/|/2'... (5 Replies)
Discussion started by: rveri
5 Replies

4. UNIX for Advanced & Expert Users

Should I say "field 8" or "column 8" in this case?

I saw some recent posts where I thought the terms "field" and "column" were being misused. I work with data a lot, and have my opinions. I'm wondering if those opinions are correct. ***** Rows seem clear - I don't think there is any controversy about what a row is, either for database or text... (10 Replies)
Discussion started by: hanson44
10 Replies

5. Shell Programming and Scripting

Read from "list1" and list matches in "list2"

I want to print any matching IP addresse in List1 with List 2; List 1 List of IP addresses; 161.85.58.210 250.57.15.129 217.23.162.249 74.76.129.101 30.221.177.237 3.147.200.59 170.58.142.64 127.65.109.33 150.167.242.146 223.3.20.186 25.181.180.99 2.55.199.32 (3 Replies)
Discussion started by: lewk
3 Replies

6. Shell Programming and Scripting

Purpose of "read" and "$END$" in ksh ?

Hi, Could anyone please shed some light on the following script lines and what is it doing as it was written by an ex-administrator? cat $AMS/version|read a b verno d DBVer=$(/usr/bin/printf "%7s" $verno) I checked that the cat $AMS/version command returns following output: ... (10 Replies)
Discussion started by: dbadmin100
10 Replies

7. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

8. UNIX for Dummies Questions & Answers

Explanation of "total" field in "ls -l" command output

When I do a listing in one particular directory (ls -al) I get: total 43456 drwxrwxrwx 2 root root 4096 drwxrwxrwx 3 root root 4096 -rwxrwxr-x 1 nobody nobody 3701594 -rwxrwxr-x 1 nobody nobody 3108510 -rwxrwxr-x 1 nobody nobody 3070580 -rwxrwxr-x 1 nobody nobody 3099733 -rwxrwxr-x 1... (1 Reply)
Discussion started by: proactiveaditya
1 Replies

9. Shell Programming and Scripting

read -p "prompt text" foo say "read: bad option(s)" in Bourne-Shell

Hallo, i need a Prompting read in my script: read -p "Enter your command: " command But i always get this Error: -p: is not an identifier When I run these in c-shell i get this error /usr/bin/read: read: bad option(s) How can I use a Prompt in the read command? (9 Replies)
Discussion started by: wiseguy
9 Replies

10. Shell Programming and Scripting

how to request a "read" or "delivered" receipt for mails

Dears, I've written a script which allows me to send mails in different formats with different attaches. Now I still want to add a feature to this script. My users would like to be able to receive a "read" or "delivered" receipt for their mails. The script send mails on behalve of an specific... (1 Reply)
Discussion started by: plelie2
1 Replies
Login or Register to Ask a Question