Add serial from second file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Add serial from second file
# 1  
Old 03-11-2015
Add serial from second file

Hi all, please consider the following scenario. I have 979 csv files in a folder each with 3 lines, first line col2 has fixed value "a", second line col2 has "b" and third line col2 has "a/b".
The first column has random names (no pattern).

filename1.csv
Code:
namep1,a
namep2,b
namec1,a/b

namefile2.csv
Code:
namep1,a
namep3,b
namec2,a/b

File3.csv
Code:
namep2,a
namep3,b
namec3,a/b

4thfile.csv
Code:
namep8,a
namep9,b
namec10,a/b

In another file there are serial numbers for each of the names. The names with "a" and "b" should have just 1 serial number, sometimes it is absent.
The names with "a/b" should always have more than 1 serial number,,sometimes absent.

Serials.csv
Code:
namep1,s1
namep3,s2
namec1,s4
namec3,s5
namec3,s6
namec3,s7
namec2,s12
namec3,s8
namec1,s9
namec1,s10
namec2,s11
namec2,s13
namec2,s14
namec1,s15

What I want to do is add serial numbers to the names in the 979 files, and when absent, just hardcode the word "absent" in the second column.
What is important here is order must be maintained, so 1st line must be "a", second line must have "b" ,,,"a/b" starts from the 3rd line.

filename1.csv.out
Code:
namep1,s1,a
namep2,absent,b
namec1,s4,a/b
namec1,s9,a/b
namec1,s10,a/b
namec1,s15,a/b

namefile2.csv.out
Code:
namep1,s1,a
namep3,s2,b
namec2,s11,a/b
namec2,s12,a/b
namec2,s13,a/b
namec2,s14,a/b

File3.csv.out
Code:
namep2,absent,a
namep3,s2,b
namec3,s5,a/b
namec3,s6,a/b
namec3,s7,a/b
namec3,s8,,a/b

4thfile.csv.out
Code:
namep8,absent,a
namep9,absent,b
namec10,absent,a/b

If I do
Code:
for f in *
do
awk -F, 'NR==FNR{a[$1]=$0; next} $1 in a { $1=a[$1]}1' Serials.csv $f > $f.out
done

... it's just pulling 1 "a/b" value. How can I get all of them?

Last edited by rbatte1; 03-11-2015 at 09:46 AM.. Reason: Tightened CODE tags to remove unecessary space int he post.
# 2  
Old 03-11-2015
If you save the following awk script in a file named Serials.awk:
Code:
BEGIN {	FS = OFS = ","
}
FNR == NR {
	n[$1,++nc[$1]] = $2
	next
}
FILENAME == "Serials.csv" {
	nextfile
}
FNR == 1 {
	if(of != "")
		close(of)
	of = FILENAME ".out"
}
{	if(!($1 in nc))
		print $1, "absent", $2 > of
	else {	print $1, n[$1, 1], $2 > of
		if(FNR == 3)
			for(i = 2; i <= nc[$1]; i++)
				print $1, n[$1, i], $2 > of
	}
}

Then the awk command:
Code:
awk -f Serials.awk Serials.csv *.csv

might be all that you need. If it gives you an argument list too long error, try the following instead:
Code:
ls | grep '.csv$' | xargs awk -f Serials.awk Serials.csv

If you want to try this on a Solaris/SunOS system, change the awk in the above commands to /usr/xpg4/bin/awk.

If the awk utility on your system does not support the nextfile function, change the line in Serials.awk that currently says:
Code:
	nextfile

to:
Code:
	next

The above script produces the output you requested, except the order of the a/b lines is different in some of the *.out files, and the line:
Code:
namec3,s8,,a/b

in File3.csv.out comes out as:
Code:
namec3,s8,a/b

which seems to be what you requested.
This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Send/receive file through serial using minicom

i have connected with my board through serial interface using minicom and i am running a bash script, which should test ethernet (ping test), USB read/write, RS232 .. I have managed to test ethernet and USB read/write. I test ethernet with ping. I test USB read/write, using dd and verifying... (10 Replies)
Discussion started by: linuxmember
10 Replies

2. Programming

Serial file transfer protocol / checksums

Hello ppl, I'm working on a windows program simulating an existing Unix program, the program is a firmware downloader, it sends a binary firmware file through the serial port (rs232) to a Set-Top Box running under unix system too, as i'm a poor unix user and not a unix programmer i choosed the... (10 Replies)
Discussion started by: nidal
10 Replies

3. Shell Programming and Scripting

file transfer using AWK over serial port

I have a unix system that only has AWK to program with which hooks up to a windows or linux based system with all progamming and software capabilities. The link between the systems is typically a 9600 baud TTY port on the AWK system and COM1 on the other system. I can cat <filenm> from the unix... (4 Replies)
Discussion started by: bobfrog
4 Replies

4. Shell Programming and Scripting

add serial number begining of each line in a text file

Dear All, i am having text file below rama 123 done raju 456 not done keshav 123 done ............... ............ i want to add a serial number to this file, the output should be 1 rama 123 done 2 raju 456 not done 3 keshav 123 done 99 ............... 100 ............ ... (3 Replies)
Discussion started by: suryanarayana
3 Replies

5. UNIX for Dummies Questions & Answers

how to add a colume with serial # in a file

I am working on a file, I need add a column with serial # in a file, the column will be the first column. and the serial # will be decide by each raw, say if I have 6 raws, I need serial # from 1 to 6 in the first column. Anyone know how to add that. I really appreciate. Thanks (2 Replies)
Discussion started by: sunsnow86
2 Replies

6. SCO

Need some advice concerning file transfer with a serial connection

Hi, I have a SCO Unix ver 5.0.7 from which I need to get some files. The problem is that I'm working remotely (have no physical access) and using Dejawin because this server is only has a serial connection to a Windows machine. Ucopy isn't on the server. What I was thinking is to do a print... (5 Replies)
Discussion started by: spartanboy184
5 Replies

7. Ubuntu

Ubuntu 9.04 Serial application to telnet to serial device

Hello! I am working on an application which reads environmental instruments which have serial ports. The application requires a serial port to be present to talk to the device (i.e. /dev/ttyS0 ). In some instances the environmental devices will be 100's of yards away from the computer, so a... (5 Replies)
Discussion started by: mvona
5 Replies

8. SCO

Serial File Transfer

I need to transfer files from a sco/unix system to a windows xp system and they are only connected via a digiboard serial connection. The terminal emulation software connects and runs a program but I need to transfer files. Anyone know a good solution? (2 Replies)
Discussion started by: chansen
2 Replies

9. Shell Programming and Scripting

redirecting serial inputs to a file?

i have an external device sending serial messages i want to connect this device to a serial port in my sun blade box and record those messages to a file, how can i read the serial port and write it to file? Thanks (4 Replies)
Discussion started by: guilartec
4 Replies

10. HP-UX

add printer on serial port

I want to add printer on a serial port can I get full details of all parameters like baudrate, parity,flow control, ... Thanks (1 Reply)
Discussion started by: Hegazy Mahdy
1 Replies
Login or Register to Ask a Question