Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Delimeters Count in a FlatFile

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Delimeters Count in a FlatFile
# 1  
Old 12-24-2010
Delimeters Count in a FlatFile
Hi,

I have the below script to check the count of delimeters for a file (here is File : test.csv Delimeter is ",")
Code:
awk '{gsub(/"[^"]*"/,x);print gsub(/,/,x)}' test.csv

And it return the output for each line as:
Code:
2
2

Code:
cat test.csv:
abc,xyz
"abc,zxyz",1

I need help one the below things:
- IS there a way to pass the delimeter as a variable ? I am using the below command but it is not working?
Code:
awk -vs="," '{gsub(/"[^"]*"/,x);print gsub(/s/,x)}' test.csv

- Can I validate a file whether it has same delimeter count across all the lines of file rather than for each line?

Please help! Thanks a lot
Last edited by Scott; 12-24-2010 at 05:35 AM.. Reason: Please use code tags
# 2  
Old 12-24-2010
Code:
awk '{FS=del; if(cnt && cnt != NF-1) {a=1;exit;} cnt=NF-1;}END{if (a==1) print "delim count is not same in all lines" else print "delim count is same in all lines"}' del=$delim test.csv

Last edited by anurag.singh; 12-24-2010 at 06:42 AM..
# 3  
Old 12-24-2010
Hi Anuraj,

The command does not solve the purpose.

Quote:
Originally Posted by anurag.singh
Code:
awk -F, '{print NF-1}' test.csv

I need to pass delimeter as a variable and at the same time want to get the output 1 or 0 if the all the rows has the same delimeter count (ignoring the delimeter in the quotes if any, that I already included in the awk command)
# 4  
Old 12-24-2010
Modified command above.
This User Gave Thanks to anurag.singh For This Post:
venkatajay_18
# 5  
Old 12-27-2010
Thanks Anuraj.

But the posted command is not giving the desired result. It always ends up going to the if(a==1) and prints the "delim count is not same in all lines" even though the delimiter count is same across the file. And also it is not ignoring the delimeter incase it is there in the quotes.

Example of such a case is:
"abc,zyz",xyz

In this case delimeter count is 1.
# 6  
Old 12-27-2010
Code:
sed 's/"\([^,]*\),\([^"]*\)"/"\1\2"/g' test.csv | awk '{FS=del; if(cnt && cnt != NF-1) {a=1;exit;} cnt=NF-1;}END{if (a==1) print "delim count is not same in all lines"; else print "delim count is same in all lines"}' del=$delim

delim is a variable, it's value should be set before running above command.
If it doesn't work, Pls post the command along with the result you got.
Last edited by anurag.singh; 12-27-2010 at 07:48 AM.. Reason: Added semiocolon in if block, also single quote at the end if sed
This User Gave Thanks to anurag.singh For This Post:
venkatajay_18
# 7  
Old 12-27-2010
Hi Anurag,

Thanks for your replies.

Here is the list of commands that I ran:
Code:
delim=,
echo $delim
output is ,

sed 's/"\([^,]*\),\([^"]*\)"/"\1\2"/g" test.csv | awk '{FS=del; if(cnt && cnt != NF-1) {a=1; exit} cnt=NF-1;}END{if (a==1)
  print "delim count is not same in all lines" else print "delim count is same in all lines"}' del=$delim

-bash: syntax error near unexpected token `{a=1'

Last edited by Scott; 12-27-2010 at 07:55 AM.. Reason: Use code tags, please...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Generating XML from a flatfile

Hi all, I am trying to generate an XML file from a flatfile in ksh/bash (could also use perl at a pinch, but out of my depth there!). I have found several good solutions on this very forum for cases where the header line in the file forms the XML tags, however my flatfile is as follows:... (5 Replies)
Discussion started by: ianmrid
5 Replies

2. Shell Programming and Scripting

sending output to flatfile

Hi, I am writing one unix script to get row count of few tables into one sequential file my script is like this $ORACLE_HOME/bin/sqlplus -s <<EOF >output.txt userid/password@databasename set verify off set heading off set feedback off select count(*) count from tablel where ; select... (4 Replies)
Discussion started by: spmsarada
4 Replies

3. Shell Programming and Scripting

Commas within Delimeters

Hi experts, I would like a favour from you guys to get the info from 5th column which was separated by the delimeter comma ( , ) The Data file is as below:- 1,USER1,"90, TEST AVENUE, OLD ROAD",test1,124,N 2,USER2,88 TEST STREET NEW ROAD,test2,123,N The User File is as below:- USER1... (1 Reply)
Discussion started by: shenkz
1 Replies

4. Shell Programming and Scripting

Finding a flatfile & deleting first line

I have a small script where I want to see if a file exists & then delete the first line from it. I have code to help me find if the file exists, but I am unsure as to how to then take in the answer and remove the first line from the flatfile: This is what I have so far just to output if the... (3 Replies)
Discussion started by: fatalxkiss
3 Replies

5. Shell Programming and Scripting

awk with two delimeters

Hi, can anyone explain me below codes...i used the below one to use two delimeter(://) at a time.. but i am not able to understand the output options.. note: here there is no requirement ..only to understand myself about below command... command: awk -F "" '{print $1}' inputfile ... (2 Replies)
Discussion started by: Shahul
2 Replies

6. Shell Programming and Scripting

Deleting column from a flatfile with delimiter

I have a set of flatfiles which have columns delimited by #. How can a particular column be deleted in all the flatfiles. All flatfiles have same number of columns. (5 Replies)
Discussion started by: rsprabha
5 Replies

7. Shell Programming and Scripting

Traverse a flatfile and check for errors

Hi, I need to check a flatfile for various parameters like length of the record, format of record, any tab character present in the record etc., for checking presence of tab character, i'm trying to use the following code and i'm not sure if the same is right. Pls Help. nawk '{print... (1 Reply)
Discussion started by: aravindc
1 Replies

8. Shell Programming and Scripting

Adding field of flatfile by an index.

Hey guys, I was given a task that involved parcing a large file that looked somthing like this... A1-0999999,SMITH,.25 A1-0999999,JOHN,.75 A1-0999999,HELMET,.1.25 A1-0999999,HOOP,.10.25 D1-1212121,SMITH,4.00 D1-1212121,TH,9.00 D1-1212121,MITCH,10.20 D1-1212121,RETAL,3.00 A1-9909555,,3.00... (2 Replies)
Discussion started by: djsal
2 Replies

9. UNIX for Dummies Questions & Answers

encrypting Unix flatfile

Hi, I am new to unix. I have a flat file that needs to be pgp encyrpted in ASCII format and ftped in Ascii format to remote location. Can any one tell me the steps involved in the pgp encryption of the unix file. I will really appreciate if any one can help me with the pgp encryption shell... (1 Reply)
Discussion started by: rkumar28
1 Replies

10. UNIX for Dummies Questions & Answers

Finding a column in a flatfile

I have a file which is fixed width columns. This is an offset buffer - rather than space or tab delimited. There are upto about 8 columns and I need to get all of the column 5's values into another file. The problem is that because the delimiter is a space - and some fields are blank - the 5th... (3 Replies)
Discussion started by: peter.herlihy
3 Replies
Login or Register to Ask a Question