Converting txt file into CSV using awk or sed


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Converting txt file into CSV using awk or sed
# 8  
Old 07-15-2012
Quote:
Originally Posted by ksk
Once I'm done with this, there is one last step, but it might be easier to explain in a separate thread or perhaps once I am to get this master CSV file.
It's never a good idea to withhold information. Better too much than too little. I've lost count of how many times I've seen someone ask for help on some very specific task, and then, later add another requirement which makes it clear that a completely different approach would have been simpler, more efficient and more maintainable. When that happens, it's a waste of time for everyone involved, both the member seeking help and the members offering help.

Whatever it is, I'm sure agama can handle it. Smilie

Regards,
Alister
These 2 Users Gave Thanks to alister For This Post:
# 9  
Old 07-15-2012
Here is a simple wrapper that reads all files in the directory and generates one csv file. Accepts the name of the output file on the command line. So if your script is files2csv and you wish the output to go to /tmp/listing.csv then the command would be: files2csv /tmp/listing.csv.

Code:
#/usr/bin/env ksh

# read the named file and output a csv of the desired data
# same awk programme  as in previous posts
function f2csv
{
    awk '
    function printit( )         # always add quotes except for the year field
    {
        if( stuff["auth"] )
        {
            sub( ",$", "",  stuff["des"] );
            printf( "\"%s\",\"%s\",\"%s\",\"%s\",%s\n", stuff["title"], stuff["auth"], stuff["source"], stuff["des"], stuff["year"] );
        }
    }

    /^Record/ { printit(); delete stuff; n = ""; next; }
    /TI:/ { n = "title"; next; }
    /DN:/ { n = "dname"; next; }
    /AU:/ { n = "auth"; next; }
    /DE:/ { n = "des"; next; }
    /PY:/ { n = "year"; next; }
    /SO:/ { n = "source"; next; }
    /^..:/ { n = ""; next; }

    n == "" { next; }

    n == "des" {
        al = split( $0, a, ";" );
        for( i = 1; i <= al; i++ )
        {
            if( substr( a[i], length(a[i]) ) == ")" )
            {
                l = split( a[i], b, " " );
                stuff[n] = stuff[n] substr( b[l], 2, length( b[l] ) - 2 ) ",";
            }
        }
        next;
    }

    { 
        sub( "^ +", "" );
        stuff[n] = stuff[n] $0 " "; next; }

    END { printit(); }
    
    ' $1
}


ls | while read file        # read all files in the directory and run them through the awk
do
    if [[ $file != $1 ]]
    then
        f2csv $file
    fi
done >>$1

Yes, all files could be given on the awk command line, but this allows some control if not all files are needed, or other checking (old/new etc.) needs to be done to refine the files actually processed.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk, sed, shell all words in INPUT.txt find in column1 of TABLE.txt and replce with column2 in

Hi dears i have text file like this: INPUT.txt 001_1_173 j nuh ]az 001_1_174 j ]esma. nuh ]/.xori . . . and have another text like this TABLE.txt j j nuh word1... (6 Replies)
Discussion started by: alii
6 Replies

2. Shell Programming and Scripting

Using awk for converting xml to txt

Hi, I have a xml script, I converted it to .txt with values comma seperated using awk function. But I want the output values should be inside double quotes My xml script (Workorders.xml) is shown like below: <?xml version="1.0" encoding="utf-8" ?> <scbm-extract version="3.3">... (8 Replies)
Discussion started by: Viswanatheee55
8 Replies

3. Shell Programming and Scripting

awk to print value from txt file to csv

Hi, I want to print two columns from a .txt file to a .csv file using awk. data in text file: Application -------------------------------------------------- ----------- OS Related Issues 1 EMEA Solutions ... (8 Replies)
Discussion started by: prashu_g
8 Replies

4. Shell Programming and Scripting

Using csh / awk / sed to compare database sizes in a txt file

Hello, I have an output file showing database sizes across the 3 environments that I use (LIVE, TEST & DEVELOPMENT). I am trying to write a script that lets me know if the size of a db on one environment is different to its corresponding db on the other environments. Here is an example... (4 Replies)
Discussion started by: stevie_g
4 Replies

5. Shell Programming and Scripting

awk/sed/something else for csv file

Hi, I have a filename.csv in which there are 3 colums, ie: Name ; prefixnumber ; number root ; 020 ; 1234567 user1,2,3 ; 070 ; 7654321 What I want is to merge colum 2 and 3 that it becomes 0201234567 or even better +31201234567 so the country number is used and drop the leading 0.... (9 Replies)
Discussion started by: necron
9 Replies

6. Shell Programming and Scripting

Converting txt file in csv

HI All, I have a text file memory.txt which has following values. Average: 822387 7346605 89.93 288845 4176593 2044589 51883 2.47 7600 i want to convert this file in csv format and i am using following command to do it. sed s/_/\./g <... (3 Replies)
Discussion started by: mkashif
3 Replies

7. Shell Programming and Scripting

Using awk/sed in handling csv file.

Please study the below script and the output Script: echo "Minimum ${host} ${process} response time=${min} ms" >> ${OUTDIR}/${OUTFILE}; echo "Maximum ${host} ${process} response time=${max} ms" >> ${OUTDIR}/${OUTFILE}; echo "Average ${host} ${process} response time=${avg} ms" >>... (0 Replies)
Discussion started by: ajincoep
0 Replies

8. UNIX for Dummies Questions & Answers

Converting txt file to csv file

Hi, Using rsync, I've sent the output to a text file. This is the text file : Pls help me on converting this text file to a csv file. Probably a script or sth to convert the text file to a csv file. (3 Replies)
Discussion started by: anaigini45
3 Replies

9. Shell Programming and Scripting

converting xls file to txt file and xls to csv

I need to convert an excel file into a text file and an excel file into a CSV file.. any code to do that is appreciated thanks (6 Replies)
Discussion started by: bandar007
6 Replies

10. Shell Programming and Scripting

AWK CSV to TXT format, TXT file not in a correct column format

HI guys, I have created a script to read 1 column in a csv file and then place it in text file. However, when i checked out the text file, it is not in a column format... Example: CSV file contains name,age aa,11 bb,22 cc,33 After using awk to get first column TXT file... (1 Reply)
Discussion started by: mdap
1 Replies
Login or Register to Ask a Question