Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Converting txt file into CSV using awk or sed
Top Forums Shell Programming and Scripting Converting txt file into CSV using awk or sed Post 302671991 by ksk on Sunday 15th of July 2012 01:54:04 PM
Old 07-15-2012
Converting txt file into CSV using awk or sed
Hello folks

I have a txt file of information about journal articles from different fields. I need to convert this information into a format that is easier for computers to manipulate for some research that I'm doing on how articles are cited. The file has some header information and then details of records. For example,
Code:
Tue Jun 19 14:07:34 EDT 2012
CSA
Database: EconLit

Record 1 of 500

DN: Database Name
    EconLit
TI: Title
    Statistical Modeling of Monetary Policy and Its Effects
AU: Author
    Sims, Christopher A
SO: Source
    American Economic Review, vol. 102, no. 4, June 2012, pp. 1187-1205
DE: Descriptors
    History of Economic Thought: Macroeconomics (B220); Economic
    Methodology (B410); Methodological Issues: General (C180); Business
    Fluctuations, Cycles (E320); Prices, Business Fluctuations, and
    Cycles: Forecasting and Simulation: Models and Applications (E370);
    Monetary Policy (E520); Modeling; Monetary; Monetary Policy; Policy
PY: Publication Year
    2012

Record 2 of 500

DN: Database Name
    EconLit
TI: Title
    Targeting the Poor: Evidence from a Field Experiment in Indonesia
AU: Author
    Alatas, Vivi; Banerjee, Abhijit; Hanna, Rema; Olken, Benjamin A;
    Tobias, Julia
SO: Source
    American Economic Review, vol. 102, no. 4, June 2012, pp. 1206-40
DE: Descriptors
    Field Experiments (C930); Measurement and Analysis of Poverty (I320);
    Welfare and Poverty: Government Programs, Provision and Effects of
    Welfare Programs (I380); Microeconomic Analyses of Economic
    Development (O120); Economic Development: Human Resources, Human
    Development, Income Distribution, Migration (O150); Economic
    Development: Urban, Rural, Regional, and Transportation Analysis,
    Housing, Infrastructure (O180); Urban, Rural, Regional, and
    Transportation Economics: Regional Migration, Regional Labor Markets,
    Population, Neighborhood Characteristics (R230); Indonesia; Asia;
    Experiment; Experiments; Field Experiment; Poor; Poverty; Village
PY: Publication Year
    2012

.
.
.

My goal is to convert this information into CSV format like so:
Code:
"TITLE","AUTHOR(S)","SOURCE","DESCRIPTOR CODES ONLY",PUBLICATION

So the above should turn into
Code:
"Statistical Modeling of Monetary Policy and Its Effects","Sims, Christopher A","American Economic Review, vol. 102, no. 4, June 2012, pp. 1187-1205","B220,B410,C180,E320,E370,E520",2012
"Targeting the Poor: Evidence from a Field Experiment in Indonesia","Alatas, Vivi; Banerjee, Abhijit; Hanna, Rema; Olken, Benjamin A; Tobias, Julia","American Economic Review, vol. 102, no. 4, June 2012, pp. 1206-40","C930,I320,I380,O120,O150,O180,R230",2012

Note that there are some descriptors that do not have codes, (eg. `Modeling' at the end of the first record). The code needs to drop those descriptors and only include the 4 character/numeric codes in brackets.

I am certain this is a fairly simple task for either awk or sed, but I don't know either with the proficiency that I should. I'd be grateful if someone can out with this.
Last edited by Scrutinizer; 07-15-2012 at 02:58 PM.. Reason: quote tags => code tags plus extra
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

AWK CSV to TXT format, TXT file not in a correct column format

HI guys, I have created a script to read 1 column in a csv file and then place it in text file. However, when i checked out the text file, it is not in a column format... Example: CSV file contains name,age aa,11 bb,22 cc,33 After using awk to get first column TXT file... (1 Reply)
Discussion started by: mdap
1 Replies

2. Shell Programming and Scripting

converting xls file to txt file and xls to csv

I need to convert an excel file into a text file and an excel file into a CSV file.. any code to do that is appreciated thanks (6 Replies)
Discussion started by: bandar007
6 Replies

3. UNIX for Dummies Questions & Answers

Converting txt file to csv file

Hi, Using rsync, I've sent the output to a text file. This is the text file : Pls help me on converting this text file to a csv file. Probably a script or sth to convert the text file to a csv file. (3 Replies)
Discussion started by: anaigini45
3 Replies

4. Shell Programming and Scripting

Using awk/sed in handling csv file.

Please study the below script and the output Script: echo "Minimum ${host} ${process} response time=${min} ms" >> ${OUTDIR}/${OUTFILE}; echo "Maximum ${host} ${process} response time=${max} ms" >> ${OUTDIR}/${OUTFILE}; echo "Average ${host} ${process} response time=${avg} ms" >>... (0 Replies)
Discussion started by: ajincoep
0 Replies

5. Shell Programming and Scripting

Converting txt file in csv

HI All, I have a text file memory.txt which has following values. Average: 822387 7346605 89.93 288845 4176593 2044589 51883 2.47 7600 i want to convert this file in csv format and i am using following command to do it. sed s/_/\./g <... (3 Replies)
Discussion started by: mkashif
3 Replies

6. Shell Programming and Scripting

awk/sed/something else for csv file

Hi, I have a filename.csv in which there are 3 colums, ie: Name ; prefixnumber ; number root ; 020 ; 1234567 user1,2,3 ; 070 ; 7654321 What I want is to merge colum 2 and 3 that it becomes 0201234567 or even better +31201234567 so the country number is used and drop the leading 0.... (9 Replies)
Discussion started by: necron
9 Replies

7. Shell Programming and Scripting

Using csh / awk / sed to compare database sizes in a txt file

Hello, I have an output file showing database sizes across the 3 environments that I use (LIVE, TEST & DEVELOPMENT). I am trying to write a script that lets me know if the size of a db on one environment is different to its corresponding db on the other environments. Here is an example... (4 Replies)
Discussion started by: stevie_g
4 Replies

8. Shell Programming and Scripting

awk to print value from txt file to csv

Hi, I want to print two columns from a .txt file to a .csv file using awk. data in text file: Application -------------------------------------------------- ----------- OS Related Issues 1 EMEA Solutions ... (8 Replies)
Discussion started by: prashu_g
8 Replies

9. Shell Programming and Scripting

Using awk for converting xml to txt

Hi, I have a xml script, I converted it to .txt with values comma seperated using awk function. But I want the output values should be inside double quotes My xml script (Workorders.xml) is shown like below: <?xml version="1.0" encoding="utf-8" ?> <scbm-extract version="3.3">... (8 Replies)
Discussion started by: Viswanatheee55
8 Replies

10. Shell Programming and Scripting

Awk, sed, shell all words in INPUT.txt find in column1 of TABLE.txt and replce with column2 in

Hi dears i have text file like this: INPUT.txt 001_1_173 j nuh ]az 001_1_174 j ]esma. nuh ]/.xori . . . and have another text like this TABLE.txt j j nuh word1... (6 Replies)
Discussion started by: alii
6 Replies
All times are GMT -4. The time now is 11:00 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy