Help in file processing


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help in file processing
# 1  
Old 04-14-2011
Network Help in file processing

Hi experts,
please help for my script..I'm confused how to do this.

i have 3 .csv file(excel file)
The 5 .csv files are

A.csv,B.csv,C.csv

A.csv has

column1 column2
a 1
b 4
c 2
d 6
e 7
f 9

B.csv has

column1 column2
a 4
b 3
c 6
d 8
e 5
f 0

C.csv has

column1 column2
a 7
b 2
c 0
d 3
e 6
f 5


i like to have final report.csv that should have the format like

column1 column2 column3 column4
a 1 4 7
b 4 3 2
c 2 6 0
d 6 8 3
e 7 5 6
f 9 0 5

The final report.csv should just display the column2 values from each .csv files side by side.

Please help me how to do this in script..i am struggling how could i do this.Thanks in advance
# 2  
Old 04-14-2011
Are they really all in the same order like that?

Code:
# open files for later
exec 5<b.csv
exec 6<c.csv
while read A B
do
        # read from b.csv
        read C D <&5
        # add more records to B
        B="$B $D"
        # read from c.csv
        read C D <&6
        B="$B $D"
        # print everything
        echo "$A $B"
done < a.csv > finalreport.csv
# close files
exec 5<&-
exec 6<&-

This User Gave Thanks to Corona688 For This Post:
# 3  
Old 04-14-2011
Bug

Are they really all in the same order like that?

yes.they are in same order...

Thanks Smilie...I'll try this.

# 4  
Old 04-16-2011
Hi corona
In .csv file the columns are comma seperated and so am getting different output rather than what i'm expected.
Can u help me in this

---------- Post updated at 08:43 AM ---------- Previous update was at 06:50 AM ----------

This code
Code:
# open files for later
exec 5<b.csv
exec 6<c.csv
while read A B
do
        # read from b.csv
        read C D <&5
        # add more records to B
        B="$B $D"
        # read from c.csv
        read C D <&6
        B="$B $D"
        # print everything
        echo "$A $B"
done < a.csv > finalreport.csv
# close files
exec 5<&-

works fine for space or tab seperated csv files..but my .csv files teh columns are "," (comma)
seperated.

so while reading from file it is reading the whole line(column1 and column2) as a
single column

my sample A.csv has
Code:
a,1

b,4

c,2

d,6

e,7

f,9

Is there any way identify the colums?

Last edited by Scott; 04-16-2011 at 07:57 PM.. Reason: Please use code tags
# 5  
Old 04-16-2011
You could look at cut and paste
Code:
cut -d, -f2 b.csv> b.tmp
cut -d, -f2 c.csv> c.tmp
paste -d, a.csv b.tmp c.tmp > final.csv

Andrew
This User Gave Thanks to apmcd47 For This Post:
# 6  
Old 04-16-2011
Never PM for tech support. It's against the rules and doesn't get me here any faster anyway. I'm not "on call".

Well of course my program's not going to work if your input's completely differently formatted than what you showed. Just a tiny change though.

Code:
# open files for later
exec 5<b.csv
exec 6<c.csv
IFS=","

while read A B
do
        # read from b.csv
        read C D <&5
        # add more records to B
        B="$B,$D"
        # read from c.csv
        read C D <&6
        B="$B,$D"
        # print everything
        echo "$A,$B"
done < a.csv > finalreport.csv
# close files
exec 5<&-
exec 6<&-

This User Gave Thanks to Corona688 For This Post:
# 7  
Old 04-16-2011
Thanks Corona688 and apmcd47.. Both worked...
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

awk - Rename output file, after processing, same as input file

I have one input file ABC.txt and one output DEF.txt. After the ABC is processed and created output, I want to rename ABC.txt to ABC.orig and DEF to ABC.txt. Currently when I am doing this, it does not process the input file as it cannot read and write to the same file. How can I achieve this? ... (12 Replies)
Discussion started by: High-T
12 Replies

2. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

3. Shell Programming and Scripting

Recursive file processing from a path and printing output in a file

Hi All, The script below read the path and searches for the directories/subdirectories and for the files. If files are found in the sub directories then read the content of the all files and put the content in csv(comma delimted) format and the call the write to xml function to write the std... (1 Reply)
Discussion started by: Optimus81
1 Replies

4. Shell Programming and Scripting

File Processing

i am having the input file as below 123456789: xxxxx12xxxxxxxxxxxxxxxxxx a_cnt 123456789: xxxxxxxxxxxxxxxxxxxxxxx a_cnt 123456789: a_cnt xxxxaq1wsxxxxxxxxxxxx12xxxxxxxxxx 123456789: xxxxxxxxxxxxasxxxx a_cnt i need the numbers in the backets of a_cnt O/p required as below 1 2 3 4... (2 Replies)
Discussion started by: expert
2 Replies

5. Shell Programming and Scripting

How to make parallel processing rather than serial processing ??

Hello everybody, I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel. My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log... (0 Replies)
Discussion started by: Samb95
0 Replies

6. Shell Programming and Scripting

How to processing the log file within certain dates based on the file name

Hi I am working on the script parsing specific message "TEST" from multiple file. The log file name looks like: N3.2009-11-26-03-05-02.console.log.tar.gz N4.2009-11-29-00-25-03.console.log.tar.gz N6.2009-12-01-10-05-02.console.log.tar.gz I am using the following command: zgrep -a --text... (1 Reply)
Discussion started by: shyork2001
1 Replies

7. Shell Programming and Scripting

how to change the current file processing to some other random file in awk ?

Hello, say suppose i am processing an file emp.dat the field of which are deptno empno empname etc now say suppose i want to change the file to emp.lst then how can i do it? Here i what i attempted but in vain BEGIN{ system("sort emp.dat > emp.lst") FILENAME="emp.lst" } { print... (2 Replies)
Discussion started by: salman4u
2 Replies

8. Shell Programming and Scripting

Checking for a control file before processing a data file

Hi All, I am very new to Shell scripting... I got a requirement. I will have few text files(data files) in a particular directory. they will be with .txt extension. With same name, but with a different extension control files also will be there. For example, Sample_20081001.txt is the data... (4 Replies)
Discussion started by: purna.cherukuri
4 Replies

9. Shell Programming and Scripting

Have a shell script check for a file to exist before processing another file

I have a shell script that runs all the time looking for a certain type of file and then it processes the file through a series of other scripts. The script is watching a directory that has files uploaded to it via SFTP. It already checks the size of the file to make sure that it is not still... (3 Replies)
Discussion started by: heprox
3 Replies
Login or Register to Ask a Question