CSV from Text file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting CSV from Text file
# 1  
Old 05-18-2018
CSV from Text file

Gents,

Attached there is a input.txt and code.txt file.

I use that code to create a csv file from txt file.

Is there the possibility to generate other code more faster to generate the CVS file using the input file.

I have deleted many data in the input file to avoid load a lot data.

Appreciate your help.

Smilie

Last edited by jiam912; 05-20-2018 at 04:16 AM.. Reason: Attached new output file
# 2  
Old 05-18-2018
For those of us who don't have gawk (or any awk with a switch statement) it would help if you would show us the output you hope to produce from the sample input you provided (file.txt).
# 3  
Old 05-19-2018
Hi Don.

I have attached the file produced with my code. ( it is a csv file )
# 4  
Old 05-19-2018
I don't understand what your code was trying to do by setting elements of your C[] array to -9999. Although not tested with the sample input you provided, I assume that you were setting elements of your V[] array to -9999 to give each field a default value if there is no line in an input record for a given field.

The following code gathers field headers from the 1st input record and gathers data from subsequent records ignoring headers (field 1 values) that were not found in the 1st record. If fields are missing from other input records the default value of -9999 will be printed as the value for those fields. The order of fields in the first input record determines the order of fields in all of the output records.

If this is what you were trying to do, the following code is considerably shorter than the code you were using and produces the same output that your code produced when given the DOS text format input file you uploaded as file.txt:
Code:
awk -F': ' '
BEGIN {	OFS = ","
}
/^___/ {if(!r++)
		for(i = 1; i <= n; i++)
			printf("%s%s", h[i], (i == n) ? ORS : OFS)
	for(i = 1; i <= n; i++) {
		printf("%s%s", v[i], (i == n) ? ORS : OFS)
		v[i] = -9999
	}
}
r == 0 {h[++n] = $1
	h2f[$1] = n
}
{	sub(/\r/, "")
	v[h2f[$1]] = $2
}' "${1:-file.txt}" > "${2:-file}.csv"

Note that if you would like to get rid of the trailing space in each field in the header output line, you can do that by changing the line:
Code:
awk -F': ' '

in the script above to:
Code:
awk -F' : ' '

Although I don't have gawk, the above code should work with gawk as well as with awk. If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.
This User Gave Thanks to Don Cragun For This Post:
# 5  
Old 05-19-2018
Don,

It is amazing and much efficient the code i did. Many thanks to improve it. Appreciate your support.

I have other small code that use same input file but select only few columns it is like this;

Code:
awk '/File/{if(s){print s;s=$NF+0}else{s=$NF+0}}
    /Swath Name/{s=s","$NF}
    /Tape #/{s=s","$NF}
    /Tape Label /{s=s","$NF}
    /Line Name/{s=s","$NF}
    /Point Number/{s=s","$NF}
    /Live Seis/{s=s","substr($0,12)}
    /SCI TB Timestamp GPS Time/{s=s","$NF}
    END{print s}' file.txt > 2.csv

Is there any change you check it. For sure you will have a better and efficient idea with this.

here is the output ( 8 selected columns by header )

Code:
5000,1000,2240,X5,40559,38741,40560:38466-39172/2(1-354) 40572:38466-39172/2(355-708) 40584:38466-39172/2(709-1062) 40596:38466-39172/2(1063-1416) 40608:38466-39172/2(1417-1770) 40620:38466-39172/2(1771-2124) 40632:38466-39172/2(2125-2478) 40644:38466-39172/2(2479-2832) 40656:38466-39172/2(2833-3186) 40668:38466-39172/2(3187-3540) 40680:38466-39172/2(3541-3894) 40692:38466-39172/2(3895-4248) 40704:38466-39172/2(4249-4602) 40716:38466-39172/2(4603-4956) 40728:38466-39172/2(4957-5310) 40740:38466-39172/2(5311-5664) 40752:38466-39172/2(5665-6018) 40764:38466-39172/2(6019-6372) 40776:38466-39172/2(6373-6726) 40788:38466-39172/2(6727-7080) 40800:38466-39172/2(7081-7434) 40812:38466-39172/2(7435-7788) 40824:38466-39172/2(7789-8142) 40836:38466-39172/2(8143-8496) 40848:38466-39172/2(8497-8850) 40860:38466-39172/2(8851-9204) 40872:38466-39172/2(9205-9558) 40884:38466-39172/2(9559-9912) 40896:38466-39172/2(9913-10266) 40908:38466-39172/2(10267-10620) 40920:38466-39172/2(10621-10974) 40932:38466-39172/2(10975-11328) 40944:38466-39172/2(11329-11682) 40956:38466-39172/2(11683-12036) 40968:38466-39172/2(12037-12390) 40980:38466-39172/2(12391-12744),1209452095780002
5001,1000,2240,X5,40549,39053,40560:38622-39484/2(1-432) 40572:38622-39484/2(433-864) 40584:38622-39484/2(865-1296) 40596:38622-39484/2(1297-1728) 40608:38622-39484/2(1729-2160) 40620:38622-39484/2(2161-2592) 40632:38622-39484/2(2593-3024) 40644:38622-39484/2(3025-3456) 40656:38622-39484/2(3457-3888) 40668:38622-39484/2(3889-4320) 40680:38622-39484/2(4321-4752) 40692:38622-39484/2(4753-5184) 40704:38622-39484/2(5185-5616) 40716:38622-39484/2(5617-6048) 40728:38622-39484/2(6049-6480) 40740:38622-39484/2(6481-6912) 40752:38622-39484/2(6913-7344) 40764:38622-39484/2(7345-7776) 40776:38622-39484/2(7777-8208) 40788:38622-39484/2(8209-8640) 40800:38622-39484/2(8641-9072) 40812:38622-39484/2(9073-9504) 40824:38622-39484/2(9505-9936) 40836:38622-39484/2(9937-10368) 40848:38622-39484/2(10369-10800) 40860:38622-39484/2(10801-11232) 40872:38622-39484/2(11233-11664) 40884:38622-39484/2(11665-12096) 40896:38622-39484/2(12097-12528) 40908:38622-39484/2(12529-12960) 40920:38622-39484/2(12961-13392) 40932:38622-39484/2(13393-13824) 40944:38622-39484/2(13825-14256) 40956:38622-39484/2(14257-14688) 40968:38622-39484/2(14689-15120) 40980:38622-39484/2(15121-15552),1209492969580002

Appreciate your help.

Last edited by Don Cragun; 05-19-2018 at 02:49 PM.. Reason: Delete duplicated text.
# 6  
Old 05-19-2018
Hi jiam912,
The code that you have provided in post #5 doesn't even come close to providing the output you showed us in that post from the sample input file named file.txt that you uploaded in post #1.

Some of the numeric fields have had trailing .0 removed, all fields have had the <carriage-return> removed from the end of the input lines, and a <space> has been removed from the output in the Live Seis field output (none of which are done by the code you showed us).

If you can't give us a clear description of the output your code is supposed to produce, I'm not going to waste any more time trying to guess at which conversions are supposed to be applied to which fields to get the output you want.

I don't have any way to know if you want the output produced by your code (which is not the output you showed us) or if you want the output you showed us (which is not the output your code produces).
# 7  
Old 05-20-2018
Hi Don,

Yes you are right, Very sorry, I forget to mention that i have removed the <carriage-return> from the output file, i did it to get it as csv file, because when i open it on excel it is not a csv file due to the <carriage-return>. Kindly check file attached named file2.zip.

it contends the 2 files

file2.csv: output after script with <carriage-return>
file3.csv: <carriage-return> removed

code

Code:
awk '/File/{if(s){print s;s=$NF+0}else{s=$NF+0}}
    /Swath Name/{s=s","$NF}
    /Tape #/{s=s","$NF}
    /Tape Label /{s=s","$NF}
    /Line Name/{s=s","$NF}
    /Point Number/{s=s","$NF}
    /Live Seis/{s=s","substr($0,12)}
    /SCI TB Timestamp GPS Time/{s=s","$NF}
    END{print s}' file.txt > file2.csv

tr -d '\r' < file2.csv > file3.csv

As you will notice, mi code greps also the line
Code:
Nb Of Live Seis : 00000

, which is not needed.

Many thanks.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Create csv from text file

Gents, I am trying to create a csv file using the file attached. I have a problem to get all information required because the rows are not continues. Here is my code till now. awk ' /"ffid"/{if(s){print s;s=$NF}else{s=$NF}} /"LineNumber"/{s=s $NF} /"PointNumber"/{s=s $NF}... (4 Replies)
Discussion started by: jiam912
4 Replies

2. Shell Programming and Scripting

Read csv file, convert the data and make one text file in UNIX shell scripting

I have input data looks like this which is a part of a csv file 7,1265,76548,"0102:04" 8,1266,76545,"0112:04" I need to make the output data should look like this and the output data will be part of text file: 7|1265000 |7654899 |A| 8|12660000 |76545999 |B| The logic behind the... (6 Replies)
Discussion started by: RJG
6 Replies

3. Shell Programming and Scripting

Process text file to create CSV

I am working on a text file where I have to get data from a text file and convert it into either CSV format or Column format as shown below. OUTPUT Expected GRP Name Pair Size DName DNumber PName PNumber adm_grp Pair1 150.00KG Pair_0ABC_1 0396 Pair_0267_s 1292 ... (6 Replies)
Discussion started by: shunya
6 Replies

4. Shell Programming and Scripting

Read in search strings from text file, search for string in second text file and output to CSV

Hi guys, I have a text file named file1.txt that is formatted like this: 001 , ID , 20000 002 , Name , Brandon 003 , Phone_Number , 616-234-1999 004 , SSNumber , 234-23-234 005 , Model , Toyota 007 , Engine ,V8 008 , GPS , OFF and I have file2.txt formatted like this: ... (2 Replies)
Discussion started by: An0mander
2 Replies

5. Shell Programming and Scripting

Converting data for text file to csv

Gents Using the script attached (raw2csv). i use to create the file .csv.. The input file is called 201.raw. Kindly can you check if there is easy way to do it. The script works fine but takes a lot time to process Thanks for your help (8 Replies)
Discussion started by: jiam912
8 Replies

6. Shell Programming and Scripting

Awk to convert a text file to CSV file with some string manipulation

Hi , I have a simple text file with contents as below: 12345678900 971,76 4234560890 22345678900 5971,72 5234560990 32345678900 71,12 6234560190 the new csv-file should be like: Column1;Column2;Column3;Column4;Column5 123456;78900;971,76;423456;0890... (9 Replies)
Discussion started by: FreddyDaKing
9 Replies

7. Shell Programming and Scripting

Conversion of spaces Text file into CSV format file

Input file (each line is separaed by spaces )given below: Name Domain Contact Phone Email Location ----------------------- ------------------------------------------------ ------- -----... (18 Replies)
Discussion started by: sreenath1037
18 Replies

8. Programming

convert text file to csv

hi all, i have a select query that gives me the output in the following way... SYSTYPE -------------------------------------------------------------------------------- Success Failures Total RFT ---------- ---------- ---------- ---------- TYP 1 0 ... (3 Replies)
Discussion started by: sais
3 Replies

9. Shell Programming and Scripting

Data fetched from text file and save in a csv file

Hi i have wriiten a script which fetches the data from text file, and saves in the output in a text file itself, but i want that the output should save in different columns. I have the output like: For Channel:response_time__24.txt 1547 data points 0.339 0.299 0.448 0.581 7.380 ... (1 Reply)
Discussion started by: rohitkalia
1 Replies

10. Shell Programming and Scripting

text file to excel or csv

Hi, I have a text file like ---------------------- aaa|bbb|ccc|ddd|eee fff|gggg|hhhhhh|kkkk -------------------- I have a script which will transfer(thourgh FTP) this text file to windows system. But I want to covert it into excel or CSF then upload into windows system.. thanks (9 Replies)
Discussion started by: srikanthus2002
9 Replies
Login or Register to Ask a Question