File creation , csv


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File creation , csv
# 8  
Old 12-23-2015
Quote:
Originally Posted by Don Cragun
If I understand your requirements correctly (and I'm not sure that I do), you could try something like:
Code:
#!/bin/ksh
f1="Object.txt"
f2="Input.txt"
f3="Output.txt"
[ -f another_object.txt ] && f4="another_object.txt" || f4="/dev/null"
awk '
FNR == 1 {
    fn++
}
{    d[fn, FNR] = $0
    if(FNR > maxline)
        maxline = FNR
}
END {    for(i = 1; i <= maxline; i++)
        print d[1, 1], (2, i) in d ? d[2,i] : "NA",
            (3, i) in d ? d[3, i] : "NA",
            (4, i) in d ? d[4, i] : "NA"
}' OFS=, "$f1" "$f2" "$f3" "$f4"

and Object.txt contains:
Code:
abcd.txt

and Input.txt contains:
Code:
abc.txt
xyz.txt

and Output.txt contains:
Code:
output_file.txt
output_file2.txt
output_file3.txt
output_file4.txt

and another_object.txt contains:
Code:
another_object_file_1.txt

produces the output:
Code:
abcd.txt,abc.txt,output_file.txt,another_object_file_1.txt
abcd.txt,xyz.txt,output_file2.txt,NA
abcd.txt,NA,output_file3.txt,NA
abcd.txt,NA,output_file4.txt,NA

but, if there is no file another_object.txt, it produces the output:
Code:
abcd.txt,abc.txt,output_file.txt,NA
abcd.txt,xyz.txt,output_file2.txt,NA
abcd.txt,NA,output_file3.txt,NA
abcd.txt,NA,output_file4.txt,NA

Note that this code will not work if any input file except the last one is an empty file and will not work if any of the input files other than another_object.txt is missing.
Thanks Don Cragun., I understand I gave little vague idea but your understanding is correct.

I cant give exact details as it would be a breach of agreement with my client.

I much appreciate your help.

Can you pls guide me how I can gain more knowledge on such AWK scripting. ( I cant understand some of the codes though Smilie )
# 9  
Old 12-23-2015
Here is a commented version of that script:
Code:
#!/bin/ksh
# Assign pathnames for 1st three files to be processed...
f1="Object.txt"
f2="Input.txt"
f3="Output.txt"
# If the 4th pathname to be proceseed is an existing regular file, use it;
# otherwise use /dev/null for the 4th file.
[ -f another_object.txt ] && f4="another_object.txt" || f4="/dev/null"

# Invoke awk to process the 4 input files...
awk '
# FNR is the record number of this line in the current file.  So the following
# action group is executed when we see the 1st line of each file we are
# processing...
FNR == 1 {	
	fn++	# Increment the input file number.
}
# Since there is no pattern on this "pattern { action }" group, it is
# executed for each input line found...
{	d[fn, FNR] = $0		# Save the data found on this input line up to,
				# but not including the line terminating
				# <newline> character ($0) in the array d[]
				# indexed by the file number and line number
				# within that file.
	if(FNR > maxline)	# If the line number is higher than we have seen
				# for any of our input files...
		maxline = FNR	# save the highest line number we have seen for
				# later use.
}
# The following section is executed after we have hit the end-of-file on all
# of our input files...
END {	# Run the following group of 1 command in a loop with i set to 1 the 1st
	# time throught the loop (i = 1), continuing while i is less than or
	# equal to the saved highest input line number (i <= maxline), and
	# incrementing i by one at the end of each pass through the loop (i++).
	for(i = 1; i <= maxline; i++) {
		# Print the following specified values on one line terminated by
		# a <newline> character each time through the loop...
		print d[1, 1],	# the data found in the 1st file on line 1
			# if there was a line number i in the 2nd file ((2, i)
			# in d), print the data found in file 2, line number i
			# (d[2, i]); otherwise print the string NA ("NA").
			(2, i) in d ? d[2,i] : "NA",
			# if there was a line number i in the 3rd file ((3, i)
			# in d), print the data found in file 3, line number i
			# (d[3, i]); otherwise print the string NA ("NA").
			(3, i) in d ? d[3, i] : "NA",
			# if there was a line number i in the 4th file ((4, i)
			# in d), print the data found in file 4, line number i
			# (d[4, i]); otherwise print the string NA ("NA").
			(4, i) in d ? d[4, i] : "NA"
	}
# End this awk script and specify that the output field separator to be used
# when printing lines is a comma (OFS=,) and the pathnames of the 4 input files
# to be processed by this script.
}' OFS=, "$f1" "$f2" "$f3" "$f4"

If anything described above still is not clear, feel free to ask more detailed questions.

The man pages for a utility are always a good place to look to see how a utility behaves. Running the command:
Code:
man awk

on your system will describe how awk works on your AIX system. Or you can look at the POSIX standard's description of awk in the POSIX section of the man pages provided on this site: POSIX - man page for awk.

Two good books describing how to use awk on Amazon.com are THE book by the authors of awk (Aho, Weinberger, and Kernighan) The AWK Programming Language (which, unfortunately, is often exorbitantly priced) and the O'Reilly book Sed & Awk.
This User Gave Thanks to Don Cragun For This Post:
# 10  
Old 12-24-2015
Thank you Don Cragun.
Your comments helped me a lot.

---------- Post updated at 12:28 PM ---------- Previous update was at 12:01 PM ----------

Code:
Slightly modified the code.
#!/bin/ksh
G=$1
f1="Object.txt"
f2="Input.txt"
f3="Output.txt"
f4="another_object.txt" ##Since this file will be always be created also can be 0 byte
awk '
FNR == 1 {
fn++
}
{ d[fn, FNR] = $0
if(FNR > maxline)
maxline = FNR
}
END { for(i = 1; i <= maxline; i++)
print "$G", d[1, i] in d ? d[1,i] : "NA", (2, i) in d ? d[2,i] : "NA",
(3, i) in d ? d[3, i] : "NA",
(4, i) in d ? d[4, i] : "NA"
}' OFS=, "$f1" "$f2" "$f3" "$f4"

How did I execute?
Quote:
ksh awk_program_name.ksh aaa.txt
Getting below error..
Quote:
awk: Function d is not defined.
The input line number is 0. The file is another_object.txt.
The source line number is 11.
# 11  
Old 12-24-2015
It would help if you showed us what output you had hoped to produce with your changes to the code I suggested. And, I'm not sure that I understand why you're getting the diagnostic that you showed us. (I don't think I've ever seen awk complain about line 0 in an input file, I don't see anything in your code where the string d is immediately followed by an open parenthesis, and I thought AIX utilities always included an error number in diagnostic messages.) But, making some guesses and assuming that you want to print the string specified by the parameter passed to your shell script as the first field in the output (instead of the 1st line in the 1st input file), you might get what you want by making the following three changes to your script:

1. Delete:
Code:
G=$1

2. Change:
Code:
awk '

to:
Code:
awk -v G="$1" '

3. And change:
Code:
print "$G", d[1, i] in d ? d[1,i] : "NA", (2, i) in d ? d[2,i] : "NA",

to:
Code:
print G, (1, i) in d ? d[1,i] : "NA", (2, i) in d ? d[2,i] : "NA",

Then try running your modified script and see if that does what you want.

Your reference to $G inside the awk script doesn't work because shell variables are not expanded inside the single quotes surrounding your awk script, double quoting a string in an awk program make the enclosed string literal, and $G in an awk program expands to the contents of the field number specified by the contents of the awk variable named G (not to the contents of the shell variable named G from the shell script that invoked awk.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Save output of updated csv file as csv file itself, part 2

Hi, I have another problem. I want to sort another csv file by the first field. result.csv SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw /home/intannf/foto5/2015_0313_090651_219.JPG,0.,-7.77223,110.37310,30.75,996.46,148.75,180.94,182.00,63.92 ... (2 Replies)
Discussion started by: refrain
2 Replies

2. Shell Programming and Scripting

Save output of updated csv file as csv file itself

Hi, all I want to sort a csv file based on timestamp from oldest to newest and save the output as csv file itself. Here is an example of my csv file. test.csv SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0739.JPG,2015:02:17 11:32:21 /home/intannf/foto/IMG_0749.JPG,2015:02:17 11:37:28... (10 Replies)
Discussion started by: refrain
10 Replies

3. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

4. Shell Programming and Scripting

Match columns from two csv files and update field in one of the csv file

Hi, I have a file of csv data, which looks like this: file1: 1AA,LGV_PONCEY_LES_ATHEE,1,\N,1,00020460E1,0,\N,\N,\N,\N,2,00.22335321,0.00466628 2BB,LES_POUGES_ASF,\N,200,200,00006298G1,0,\N,\N,\N,\N,1,00.30887539,0.00050312... (10 Replies)
Discussion started by: djoseph
10 Replies

5. Shell Programming and Scripting

Fixed Width file creation from csv

Hello All, I'm able to achieve my goal of creating a fixed width file from a comma delimited but I know I'm not doing it as efficiently as possible. Original File checksab 004429876883,O,342040,981.98,10232014 004429876883,O,322389,2615.00,10232014... (6 Replies)
Discussion started by: aahlrich
6 Replies

6. Shell Programming and Scripting

CSV File Creation Within Shell Script

Hi All, I am trying to create a CSV file within a shell script test.ksh and the code snippet is something like below: #!/usr/bin/ksh # Set required variables. . $HOME/.prof # Output file path Group1=/tmp/G1.csv Group2=/tmp/G2.csv Group3=/tmp/G3.csv $ORACLE_HOME/bin/sqlplus -s... (2 Replies)
Discussion started by: swasid
2 Replies

7. Shell Programming and Scripting

Comparing 2 CSV files and sending the difference to a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 ... (1 Reply)
Discussion started by: Naresh101
1 Replies

8. Shell Programming and Scripting

2 problems: Mailing CSV file / parsing CSV for display

I have been trying to find a good solution for this seemingly simple task for 2 days, and I'm giving up and posting a thread. I hope someone can help me out! I'm on HPUX, using sqlplus, mailx, awk, have some other tools available, but can't install stuff that isn't already in place (without a... (6 Replies)
Discussion started by: soldstatic
6 Replies

9. UNIX for Dummies Questions & Answers

csv file creation

goodpeople... i have following script which collects info from tab and then generates csv file. all is good except 3 digt and 4 digit enties from tab Issue here is that csv file is not okay with 3 digit and 4 digit entries that reside in tab. script... #!/bin/ksh # # Script to send... (2 Replies)
Discussion started by: Student37
2 Replies
Login or Register to Ask a Question