Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Shell script to extract data from csv file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script to extract data from csv file
# 1  
Old 09-03-2013
Shell script to extract data from csv file
Hi everyone,

I have a csv file which has data with different heading and column names as below.

Code:
 
Static Data Ingested
,,,,,,,,,,,,Known Explained Rejections
Column_1,column_2,Column_3,Column_4,,Column_6,Column_7,,% Column_8,,Column_9 ,Column_10 ,
viv,Public_456,08/28/2013,1234566,,08/31/2013,5896325,,0.00,,0,0
SDS,Public_ddd,08/28/2013,589652,,08/31/2013,2365896,,0.00,,0,0

Code:
 
Static Data extracted
Column_1,Column_2,Column_3,Column_4,,Column_6
rms,k2c,08/28/2013,1234566,08/31/2013,5896325,0.00,0,0
SDS,k3c,08/28/2013,589652,08/31/2013,2365896,0.00,0,0


In the first one there are 10 columnwhere columns are separated by either , or ,,
Static Data Ingested is the heading for first data.

Similarily for second Static Data extracted is the heading and has 6 columns all separated by ,.

i want to write a shell script to extract 1 and 6-10 columns from first one and 1 and 4-6 from second.
Please help.
# 2  
Old 09-03-2013
In the first section you could use ",*" ad the field separator and "," in the second section, but the number of columns does not appear to be 10 in the first section, not 6 in the second section.

Could you give a sample of the output?
# 3  
Old 09-03-2013
Try this (but I'm afraid your input files' structure is somewhat inconsistent):
Code:
awk 'FNR==1 {f++; next} f==1 {print $1, $6, $7, $8, $9, $10} f==2 {print $1, $4, $5, $6}' OFS=, FS=",+" file1 file2
,,,,,
Column_1,Column_7,% Column_8,Column_9,Column_10 ,
viv,5896325,0.00,0,0,
SDS,2365896,0.00,0,0,
Column_1,Column_4,Column_6,
rms,1234566,08/31/2013,5896325
SDS,589652,08/31/2013,2365896

# 4  
Old 09-03-2013
Sample output:

Code:
Static Data Ingested:
Column_1,Column_6,Column_7,,% Column_8,,Column_9 ,Column_10
viv           5896325,  0.00      ,,    0          ,,0            ,1

Static Data extracted
Column_1,Column_2,Column_3,Column_4,Column_6
rms,k2c,08/28/2013,1234566,08/31/2013,5896325,0.00,0,0
SDS,k3c,08/28/2013,589652,08/31/2013,2365896,0.00,0,0

# 5  
Old 09-03-2013
That output can't be obtained from your sample input files with the specifications gieven in post#1; e.g. if the field separator is allowed to be ;;. Where's "Known Explained Rejections"? How come $1, $4, $5, and $6 are more that four columns?
# 6  
Old 09-03-2013
Agree with RudiC
I think you are confused about the format of the input file. Also, your sample output doesn't make any sense with your original request.

--ahamed
# 7  
Old 09-03-2013
ok..even if the files are only , separated how can i extract the output as below:

Code:
Static Data Ingested:
Column_1,Column_6,Column_7,% Column_8,Column_9 ,Column_10
viv           5896325,  0.00      ,    0          ,0            ,1

Static Data extracted:

Column_1,Column_2,Column_3,Column_4,Column_6
rms,k2c,08/28/2013,1234566,08/31/2013,5896325,0.00,0,0
SDS,k3c,08/28/2013,589652,08/31/2013,2365896,0.00,0,0

where Static Data Ingested and Static Data extracted are theheadings of the respective records.
Last edited by Scrutinizer; 09-03-2013 at 08:13 PM.. Reason: code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Shell script to extract data in a file

I have this 5GB file, and i want to extract from the file particulars pattern. this is my script: // count=`grep -wc "MSISDN" file_name` k=1 >OUTPUT >OUTPUT_Final while do cat file_name | awk -F":" -v var="$k" '$1=="MSISDN" {m++}m==var{print; exit}' >> OUTPUT cat file_name |awk -F":"... (33 Replies)
Discussion started by: gillesi
33 Replies

2. UNIX for Dummies Questions & Answers

Shell script to extract data from csv file

Hi Guys, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 7 columns having values say column 1,column 2.....column 7 as below along with their values. Name, Address,... (7 Replies)
Discussion started by: Vivekit82
7 Replies

3. UNIX for Dummies Questions & Answers

Shell script to extract data from csv file based on certain conditions

Hi Guys, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (1 Reply)
Discussion started by: Vivekit82
1 Replies

4. Shell Programming and Scripting

Read data from .csv file through shell script & modify

I need to read data from a file called "test.csv" through shell script where the file contains values like name,price,descriptor etc. There are rows where descriptor (& in some rows name) are written as string & other characters like "car_+" OR "bike*" etc where it should contains strings like... (3 Replies)
Discussion started by: raj100
3 Replies

5. Shell Programming and Scripting

How to extract data from csv file

Hello everybody, Here is my problem, I don't know anything about shell programming and my boss is actually asking me to develop a shell script in order to get values in a csv file from a specific date. Here is a sample of the csv file : Date;Enchaînement;Titre;Libellé ;calendrier;Heure début;Heure... (11 Replies)
Discussion started by: freyr
11 Replies

6. Shell Programming and Scripting

need a shell script to extract data from a log file.

If I have a log like : Mon Jul 19 05:07:34 2010; TCP; eth3; 52 bytes; from abc to def Mon Jul 19 05:07:35 2010; UDP; eth3; 46 bytes; from aaa to bbb Mon Jul 19 05:07:35 2010; TCP; eth3; 52 bytes; from def to ghi I will need an output like this : Time abc to def... (1 Reply)
Discussion started by: hitha87
1 Replies

7. Shell Programming and Scripting

Exporting data as a CSV file from Unix shell script

Friends...This is the first time i am trying the report generation using shell script... any suggestions are welcome. Is there a way to set the font size & color when i am exporting the data from unix shell script as a CSV file ? The following sample data is saved as a .csv file in the... (2 Replies)
Discussion started by: appu2176
2 Replies

8. Shell Programming and Scripting

Help with shell script to extract data from XML file

Hello Scripting Gurus, I need help with extracting data from the XML file using shell script. The data is in a large XML and I need to extract the id values of all completedworkflows. Here is a sample of it. Input and output data is also in the attached text files. <wfregistry>... (5 Replies)
Discussion started by: yajaykumar
5 Replies

9. Shell Programming and Scripting

shell-script which extract data from log file

give me a shell-script which extract data from log file on a server by giving date and time as input (for both start time and end time) and it will give the logs generated during the given time as output. (4 Replies)
Discussion started by: abhishek27
4 Replies

10. Shell Programming and Scripting

Shell Script to Load data into the database using a .csv file and .ctl file

Since i'm new to scripting i'm findind it difficult to code a script. The script has to be an executable with 2 paramters passed to it.The Parameters are 1. The Control file name(.ctl file) 2. The Data file name(.csv file) Does anybody have an idea about it? :confused: (3 Replies)
Discussion started by: Csmani
3 Replies
Login or Register to Ask a Question