Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Cutting a file with multiple delimiters into columns

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Cutting a file with multiple delimiters into columns
# 1  
Old 03-31-2010
Cutting a file with multiple delimiters into columns
Hi All

I have recently had to start using Unix for work and I have hit brick wall with this prob....

I have a file that goes a little something like this....

Code:
EUR;EUR;EUR:USD:USD;USD;;;EUR/USD;XAU/AUD;XAU/EUR;XAU/AUD,GBP/BOB,UAD/XAU;;;1.11;2.22;3.33;4.44;5.55;6.66;;;


is it possible to create a script to manipulate this so that it reads each line the once it hits the ;;; begin a new line?

desired output being...
Code:
EUR          EUR          EUR           USD           USD          USD
EUR/USD      XAU/AUD      XAU/EUR       XAU/EUR       GBP/BOB      UAD/XAU
1.11         2.22         3.33          4.44          5.55         6.66

and furthermore, use that script to accept a parameter eg "EUR"
and have it output the columns that the world matches....like a grep by column if you will

Thanks a mil in advance...

LC
Last edited by Scott; 03-31-2010 at 07:56 PM.. Reason: Please use code tags
# 2  
Old 04-01-2010
Code:
awk -F ';;;' '
{
  gsub(";","\t",$1)
  gsub("[;,]","\t",$2)
  gsub(";","\t",$3)
  print $1
  print $2
  print $3
}
' "$file"

# 3  
Old 04-01-2010
Code:
awk -F";;;" -v w=15 ' {
        for ( i = 1; i <=NF; i++ ){
                split ( $i, s, ";" )
                for ( si in s ){
                        if ( s[si] ~ ":|," ){
                                char = (s[si] ~ ",") ?  "," : ":"
                                split( s[si], c, char )
                                for ( ci in c )
                                        printf("%-"w"s", c[ci])
                        }
                        else
                                printf("%-"w"s", s[si])
                }
                printf("\n")
        }
}' $file

Output: -

Code:
EUR            EUR            EUR            USD            USD            USD            
EUR/USD        XAU/AUD        XAU/EUR        XAU/AUD        GBP/BOB        UAD/XAU        
1.11           2.22           3.33           4.44           5.55           6.66

# 4  
Old 04-01-2010
Or maybe with:
Code:
awk -F"[:;,]" '$1=$1' RS=";;;" OFS="\t"

# 5  
Old 04-01-2010
Code:
$ sed 's/;;;/\
/g; s/[,;:]/____/g; s/\n$//' data
EUR     EUR     EUR     USD     USD     USD
EUR/USD XAU/AUD XAU/EUR XAU/AUD GBP/BOB UAD/XAU
1.11    2.22    3.33    4.44    5.55    6.66

*The red underscore should be a literal tab
# 6  
Old 04-02-2010
To get the columns: -

Code:
awk -F";;;" -v w=15 -v m="EUR" ' {
        out = ""
        for ( i = 1; i <=NF; i++ ){
                col = 0
                split ( $i, s, ";" )
                for ( si in s ){
                        if ( s[si] ~ ":|," ){
                                char = (s[si] ~ ",") ?  "," : ":"
                                split( s[si], c, char )
                                for ( ci in c ) {
                                        out = out sprintf("%-"w"s", c[ci])
                                        col++
                                        if ( c[ci] ~ m )
                                                _[col]++
                                }       
                        }
                        else {
                                out = out sprintf("%-"w"s", s[si])
                                col++
                                if ( s[si] ~ m )
                                        _[col]++
                        }       
                }
                printf("%s\n", out)
                out = ""
        }
        }END{
                out = m" is found in columns "
                for ( i in _ )
                        out = out sprintf("%d%c", i, ",")
                printf("%s\n", substr(out, 1, length(out) -1))
} ' y

Output: -

Code:
EUR            EUR            EUR            USD            USD            USD            
EUR/USD        XAU/AUD        XAU/EUR        XAU/AUD        GBP/BOB        UAD/XAU        
1.11           2.22           3.33           4.44           5.55           6.66           

EUR is found in columns 1,2,3



---------- Post updated 02-04-10 at 09:08 AM ---------- Previous update was 01-04-10 at 11:07 PM ----------
Quote:
Originally Posted by Franklin52
Or maybe with:
Code:
awk -F"[:;,]" '$1=$1' RS=";;;" OFS="\t"

That's nice Franklin. What was your thinking with $1=$1? I don't understand it.

Did you notice $2=$2 removes the extra blank lines from the output?
# 7  
Old 04-02-2010
MySQL
additional alister solution's as alternative

Code:
sed -e 's/;;;/\n/g' -e 's/[;,:]/\t/g' file

Regards
Yucel
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Getting fields from a file having multiple delimiters

Hi All, I have a file with a single row having the following text ABC.ABC.ABC,Database,New123,DBNAME,F,ABC.ABC.ABC_APP,"@FUNCTION1("ENT1") ,@FUNCTION2("ENT2")",R, I want an output in the following format ABC.ABC.ABC DBNAME ABC.ABC.ABC_APP '@FUNCTION1("ENT1")... (3 Replies)
Discussion started by: dev.devil.1983
3 Replies

2. UNIX for Dummies Questions & Answers

Cutting specific columns from lines

I am trying to remove columns 81-97 from a line that can be as long as 114 characters. Because a number of lines might not have under 80 characters, using the cut command following by paste could be a problem. While sed might work, is there some other utility that could do this more easily? ... (9 Replies)
Discussion started by: wbport
9 Replies

3. UNIX for Dummies Questions & Answers

Seperate columns according to delimiters

Hi all I need your help to separate colomns based on "-" delimiter for a very big file 30 millions rows I have a colmun looking like this : clomun 1 1-100000989-A_ATC 1-10000179-AAAAA 1-100002154-TGTTA 1-100002155-GTTAG 1-100002443 1-100002490 1-100002619 I need to separte in three... (5 Replies)
Discussion started by: biopsy
5 Replies

4. Shell Programming and Scripting

Awk match multiple columns in multiple lines in single file

Hi, Input 7488 7389 chr1.fa chr1.fa 3546 9887 chr5.fa chr9.fa 7387 7898 chrX.fa chr3.fa 7488 7389 chr21.fa chr3.fa 7488 7389 chr1.fa chr1.fa 3546 9887 chr9.fa chr5.fa 7898 7387 chrX.fa chr3.fa Desired Output 7488 7389 chr1.fa chr1.fa 2 3546 9887 chr5.fa chr9.fa 2... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

5. UNIX for Dummies Questions & Answers

cutting multiple columns into multiple files

Hypothetically, suppose that file1 id v1 v2 v3 v4 v5 v6 v7..........v100 1 1 1 1 1 1 2 2 .....50 2 1 1 1 1 1 2 2 .....50 3 1 1 1 1 1 2 2 .....50 4 1 1 1 1 1 2 2 .....50 5 1 1 1 1 1 2 2 .....50 I want to write a loop such that I take the id# and the first 5 columns (v1-v5) into the... (3 Replies)
Discussion started by: johnkim0806
3 Replies

6. Shell Programming and Scripting

Sorting problem: Multiple delimiters, multiple keys

Hello If you wanted to sort a .csv file that was filled with lines like this: <Ticker>,<Date as YYYYMMDD>,<Time as H:M:S>,<Volume>,<Corr> (H : , M, S: ) by date, does anybody know of a better solution than to turn the 3rd and 4th colons of every line into commas, sorting on four keys,... (20 Replies)
Discussion started by: Ryan.
20 Replies

7. UNIX for Dummies Questions & Answers

cutting columns if delimiter has more than one charecter (|^)

Hi All, I am having a file with the delimiter '|^'. File name:test_dlim.csv I want to cut the first field of this using awk command. I tried with the help of the following link:... (2 Replies)
Discussion started by: boopathyvasagam
2 Replies

8. Shell Programming and Scripting

Cutting columns starting at the end of each line...

Hi Guys, Can you help me with a sed or a csh script that will have an output from the input below. Cutting the columns starting from the end of the line and not from the start of the line? Sample1 - The underscore character "_" is actually a space...i need to put it as underscore here coz... (2 Replies)
Discussion started by: elmer1503
2 Replies

9. Shell Programming and Scripting

Cutting Columns and Moving in to a file

Guys, Can any one tell me how can we cut the columns and move each column in to a separate file using awk? I have a tab delimited file as shown below, 1213 wattt werree 2345 skhasdjh aasas I want to output this in to three files named a.txt,b.txt and c.txt say a.txt... (3 Replies)
Discussion started by: Serious Sam
3 Replies

10. UNIX for Dummies Questions & Answers

cutting columns if delimiter has more than one charecter

Hi, My file looks like abc$%sdfhs$%sdf$%sdfaf$% here as seen delimiter is $%...now how cas i take out second field as cut command expect delimiter as single charecter only.....is there is any other way thanks and regards mahabunta (9 Replies)
Discussion started by: mahabunta
9 Replies
Login or Register to Ask a Question