Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Count delimiter and replace

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Count delimiter and replace
# 1  
Old 10-21-2010
Count delimiter and replace
Hi,
I have below sample data file. I want to count the delimiter no of times repeated and replace with new line characters. The new line characters appear somtimes in between of the records and end of the record:

input file:
Code:
jack;1500;manager;boston\n
jim;2000;\n
developer;texas\n
bob;5000;director\n
;atlanta\n

Expected output file:
Code:
jack;1500;manager;boston\n
jim;2000;developer;texas\n
bob;5000;director;atlanta\n

I am new to shell scripting. How can I achieve this. The parameter of script should be delimeter, file name and count repetition.

Thanks in advance.

-subha

Moderator's Comments:
Mod Comment Use [code] and [/code] tags when posting code, data or logs to preserve formatting and enhance readability, thanks
Last edited by zaxxon; 10-21-2010 at 02:11 AM..
# 2  
Old 10-21-2010
I'm sure there is a better way to do this but here is my quick solution.

Code:
delimit=$1
file=$2
count=$3
sed "s/${delimit}\$//; s/^${delimit}//" $file | awk -F${delimit} -vT=$count '
{ 
    RS=""
    for(i=1; i <= NF; i++ ) {
        printf $i
        printf i % T ? FS : "\n"
    }
}'

Called like this';

Code:
$ ./solution ";" inputfile 4
jack;1500;manager;boston
jim;2000;developer;texas
bob;5000;director;atlanta

# 3  
Old 10-24-2010
Thanks for the script.
I am hitting with following error msg when I am processing files that has 40k lines.

Error msg:
^ ran out for this one

Is there any way we can fix the above error msg. I want to generate output file and turn off the screen output when I run the script.
# 4  
Old 10-24-2010
Code:
echo -e "$(tr -d '\n' <input | sed 's:;\\n:;:g;s:\\n;:;:g;s:\\n$::')"

Code:
[ctsgnb@shell ~]$ cat input
jack;1500;manager;boston\n
jim;2000;\n
developer;texas\n
bob;5000;director\n
;atlanta\n
[ctsgnb@shell ~]$ echo -e "$(tr -d '\n' <input | sed 's:;\\n:;:g;s:\\n;:;:g;s:\\n$::')"
jack;1500;manager;boston
jim;2000;developer;texas
bob;5000;director;atlanta
[ctsgnb@shell ~]$

Last edited by ctsgnb; 10-24-2010 at 07:59 PM.. Reason: little fix to remove ending blank line
# 5  
Old 10-24-2010
Code:
awk -F \; '{for (i=1;i<=NF;i++) {if ($i!="") printf (++j%4)?$i FS:$i RS}}' infile

This User Gave Thanks to rdcwayx For This Post:
Chubler_XL
# 6  
Old 10-24-2010
Nice solution rdcwayx - I know there had to be a better way to do it. Only thing to look out for is blank fields (not sure if they are allowed or not).
# 7  
Old 10-25-2010
Tr this,
Code:
awk  -F";" '{if (NF == 4) {print $0} else {printf substr($0,1,(length($0)-2));getline;print $0}}' inputfile

Last edited by pravin27; 10-25-2010 at 03:47 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split files based on row delimiter count

I have a huge file (around 4-5 GB containing 20 million rows) which has text like: <EOFD>11<EOFD>22<EORD>2<EOFD>2222<EOFD>3333<EORD>3<EOFD>44<EOFD>55<EORD>66<EOFD>888<EOFD>9999<EORD> Actually above is an extracted file from a Sql Server with each field delimited by <EOFD> and each row ends... (8 Replies)
Discussion started by: amvip
8 Replies

2. Shell Programming and Scripting

Linux shell script to insert new lines based on delimiter count

The input file is a .dat file which is delimited by null (^@ in Linux). On a windows PC it looks something like this (numbers are masked with 1). https://i.imgur.com/nta2Gqp.jpg The entire file is in one row but it has multiple records - each record contains 80 fields i.e. there are 81 counts... (9 Replies)
Discussion started by: digitalnirvana
9 Replies

3. Shell Programming and Scripting

awk delimiter count if mismatched?

Hi all, I have a file where in it has lot of records in it. I have written below stuff to find the number of fields as shown below `awk -F '|' '{print NF-1}' file.txt| head -1` how do i proceed if in case any record in particular is having more number of delimiters, if it having??? what... (7 Replies)
Discussion started by: nikhil jain
7 Replies

4. Shell Programming and Scripting

Replace specific column delimiter

Hi All, I have a file with a pipe delimiter. I need to replace the delimiter with html tags. I managed to get all the delimiters replaced along with first and last but the requirement is that I need to change 7th delimiter with slight change. File1: ... (2 Replies)
Discussion started by: shash
2 Replies

5. Shell Programming and Scripting

Count delimiter(~|*) each row in a file and return 1 or 0

Hi I want to check delimiter in file. Delimiter in my file is ~|* sample of file : ABC~|*edgf~|*T1J333~|*20121130 ABC~|*sdaf~|*T1J333~|*20121130 ABC~|*fsdg~|*T1J333~|*20121130 ABC~|*dfsg~|*T1J333~|*20121130 in this i want to count number delimiter occur is 4 in each row if count is... (21 Replies)
Discussion started by: MOHANP12
21 Replies

6. Shell Programming and Scripting

Skip the delimiter with in double quotes and count the number of delimiters during data extract

Hi All, I'm stuck-up in finding a way to skip the delimiter which come within double quotes using awk or any other better option. can someone please help me out. Below are the details: Delimited: | Sample data: 742433154|"SYN|THESIS MED CHEM PTY.... (2 Replies)
Discussion started by: BrahmaNaiduA
2 Replies

7. UNIX for Dummies Questions & Answers

How to replace particular occurrence of character in between a delimiter?

Hi, Hi, I have a file with following format 1|" "text " around " |" fire "guest"|" " 2| "xyz"" | "no guest"|"3" 3| """ test3""| "one" guest"|"4" My requirement is to replace all occurrences of " to ' which are occurring between |" "|delimiter so my output should look like this 1|"... (3 Replies)
Discussion started by: H_bansal
3 Replies

8. Shell Programming and Scripting

Delimiter count

Hi I wrote shell script for count the deleimeters and send it to reject file which are exeeseding number of count. But script is not working properly... Can you please check the below script and modify it as required. ex : 1,abc 2,mno 3,pqr 4,xyz,error As per the requiredment last... (3 Replies)
Discussion started by: victory
3 Replies

9. Shell Programming and Scripting

Delimiter count in a string.

Hi , I need to get the delimiter "-" count in a particular string. string= SYS_NAME-123-S5-2008-10-20.LOG the delimit "-" count is 5 . Using sed or awk can I know the count ? I have seen how to get the count for delimiter in a file but not a string :( Thanks, Priya (8 Replies)
Discussion started by: priyam
8 Replies

10. Shell Programming and Scripting

replace the last delimiter

hi, i have a file of the below format: WA| 3|52|Waste | PW| 3|53|Promotion Waste | BR| 3|54|Breakage | TH| 3|55|Theft | IC| 3|56|Insurance Claim | I want to replace the last delimiter | by using sed command ...can u please help me witht this Thanks in... (4 Replies)
Discussion started by: jisha
4 Replies
Login or Register to Ask a Question