Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help with pipe count in a flat file!!!

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with pipe count in a flat file!!!
# 1  
Old 06-01-2007
Help with pipe count in a flat file!!!
Hi Friends,

I have a flat file and it has 43 columns which means 42 pipes but some records have less number of pipes.Can anyone tell me a command to count the number of pipes in a record and redirect the count to some new file because the flat file has not less than 40,000 records.

Thanks for your help,
KUmar
# 2  
Old 06-01-2007
there's more than one way:
Code:
awk -F "|" '{print NF-1}' file > newfile

# 3  
Old 06-01-2007
Quote:
Originally Posted by ghostdog74
there's more than one way:
Code:
awk -F "|" '{print NF-1}' file > newfile

> cd /space/dwland/shell_prg_test/Trans/
> awk -F "|" '{print NF-1}' Daily_Feed_051507.txt > newfile
awk: syntax error near line 1
awk: bailing out near line 1
>

It is giving syntax error...I'm using Sun OS 5.10...Am i doing something wrong here...Could you please tell,the way to run the command....
# 4  
Old 06-01-2007
i don't have SunOS for testing. you can try using nawk.
# 5  
Old 06-01-2007
Kumar,
See if this works for you:
Code:
typeset -i mCnt=1
mRegExp='.*\|'
mFullRE=""
while [ $mCnt -le 42 ]
do
  mFullRE=${mFullRE}${mRegExp}
  mCnt=${mCnt}+1
done
egrep -v "${mFullRE}" input_file

# 6  
Old 06-01-2007
The two following commands extract records with a number of fields different of 43 ;
Code:
egrep -v '^([^|]*\|){42,42}[^|]*$' inputfile > invalid_records_file
sed   '/^\([^|]*|\)\{42\}[^|]*$/d' inputfile > invalid_records_file

If you want to display the number of fields for invalid records, the best way is awk (or nawk).
Code:
awk -F'|' 'NF!=43 {print "Record",NR,"Fields count:",NF}' infile > invalid_records_list

Jean-Pierre.
# 7  
Old 06-01-2007
Aigles,
I replaced the 42 by 2 in your solutions and tried both of them,
"egrep" and "sed", and neither one worked.
Could you please advise?
Here is my set of data:
Code:
abc
abc|
abc|def|
abc|def|123|
abc|def|123|456|
abc|def|123|456|xyz|
abc|def|123|456|xyz
abc|def|123|456
abc|def|123
abc|def
abc
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Oracle table extract: all columns are not converting into pipe delimited in flat file

Hi All, I am writing a shell script to extract oracle table into a pipe dilemited flat file. Below is my code and I have attached two files that I have abled to generate so far. 1. Table.txt ==> database extract file 2. flat.txt ==> pipe delimited after some manipulation of the original db... (5 Replies)
Discussion started by: express14
5 Replies

2. Shell Programming and Scripting

How to ignore Pipe in Pipe delimited file?

Hi guys, I need to know how i can ignore Pipe '|' if Pipe is coming as a column in Pipe delimited file for eg: file 1: xx|yy|"xyz|zzz"|zzz|12... using below awk command awk 'BEGIN {FS=OFS="|" } print $3 i would get xyz But i want as : xyz|zzz to consider as whole column... (13 Replies)
Discussion started by: rohit_shinez
13 Replies

3. Shell Programming and Scripting

Unique count from flat file

Hello Guys I have a flat file with '|~|' delimited When I use to record count using below command awk -FS"+" ' {print $colno}' filename | wc -l the count is fine But when I am trying to find the unique number of record the o/p is always 1 awk -FS"+" ' {print $colno}'... (11 Replies)
Discussion started by: Pratik4891
11 Replies

4. Shell Programming and Scripting

count of null in pipe delimited txt file

Hi, I have a pipe delimited txt file which contains 17 fields per line/row. 16th field contains email id. I want to count the number of lines/rows that contains null in the 16th field. Plz find attached example data file. I'm looking for a command line/script which achieves this. ... (5 Replies)
Discussion started by: Sriranga
5 Replies

5. Shell Programming and Scripting

Searching for Log / Bad file and Reading and writing to a flat file

Need to develop a unix shell script for the below requirement and I need your assistance: 1) search for file.log and file.bad file in a directory and read them 2) pull out "Load_Start_Time", "Data_File_Name", "Error_Type" from log file 4) concatinate each row from bad file as... (3 Replies)
Discussion started by: mlpathir
3 Replies

6. UNIX for Dummies Questions & Answers

Grep char count & pipe to sed command

Hi I am having a 'grep' headache Here is the contents of my file: (PBZ,CP,(((ME,PBZ,BtM),ON),((ME,((PBZ,DG),(CW9,PG11))),CW9,TS2,RT1))) I would like to count out how many times 'PBZ' occurs and then place that number in the line above 3... (8 Replies)
Discussion started by: cavanac2
8 Replies

7. Shell Programming and Scripting

count alphabets in a flat file and print

I have a text file with a huge dataset, and each row in that dataset has some data(65479 rows). In that file I need to find the number of times a-z & A-Z Appears. How Can I Initialise Array into an Array to parse the count also I need to parse a,A into a single array preferably. example of... (3 Replies)
Discussion started by: vmsenthil
3 Replies

8. Shell Programming and Scripting

Getting Sum, Count and Distinct Count of a file

Hi all this is a UNIX question. I have a large flat file with millions of records. col1|col2|col3 1|a|b 2|c|d 3|e|f 3|g|h footer**** I am supposed to calculate the sum of col1 1+2+3+3=9, count of col1 1,2,3,3=4, and distinct count of col1 1,2,3=c3 I would like it if you avoid... (4 Replies)
Discussion started by: singhabhijit
4 Replies

9. Programming

compare XML/flat file with UNIX file system structure

Before i start doing something, I wanted to know whether the approach to compare XML file with UNIX file system structure. I have a pre-configured file(contains a list of paths to executables) and i need to check against the UNIX directory structure. what are the various approches should i use ? I... (6 Replies)
Discussion started by: shafi2all
6 Replies

10. UNIX for Dummies Questions & Answers

How to count the record count in an EBCDIC file.

How do I get the record count in an EBCDIC file on a Linux Box. :confused: (1 Reply)
Discussion started by: oracle8
1 Replies
Login or Register to Ask a Question