Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

adding a column of unique string to multiple files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting adding a column of unique string to multiple files
# 1  
Old 01-29-2012
adding a column of unique string to multiple files
Hi, I need to add a column of unique string to individual csv file.

my individual csv file looks like this and each file has variable row length:
Code:
178.52,39.4,train,0.003355544375334351,39.15752753933254,0.4895933968953914
178.75,39.15,train,0.0015295405476648673,23.774272858698644,0.3042207089509684
178.47,39.37,train,0.002269213484111252,31.340746907207915,0.39345477749708396
175.45,40.52,train,0.010551857362453402,72.18189851536339,0.7510192047818629
175.02,40.4,train,0.008422155518928283,63.40264697097201,0.7065360359559728
175.37,40.48,train,0.003194810966554387,37.51552339023896,0.47733591157345767

and I want to add string and end up with an output like this:
Code:
199906_sqd,178.52,39.4,train,0.003355544375334351,39.15752753933254,0.4895933968953914
199906_sqd,178.75,39.15,train,0.0015295405476648673,23.774272858698644,0.3042207089509684
199906_sqd,178.47,39.37,train,0.002269213484111252,31.340746907207915,0.39345477749708396
199906_sqd,175.45,40.52,train,0.010551857362453402,72.18189851536339,0.7510192047818629
199906_sqd,175.02,40.4,train,0.008422155518928283,63.40264697097201,0.7065360359559728
199906_sqd,175.37,40.48,train,0.003194810966554387,37.51552339023896,0.47733591157345767

Is it possible to do a loop to do this routine?
I have a total of 42 csv files and an array of unique strings to be added on each csv file include the ones below:
Code:
spname=(199906_sqd 199907_sqd 199908_sqd 199909_sqd 199910_sqd 200005_sqd 200006_qd 200007_sqd \
200008_sqd 200009_sqd 200010_sqd 200011_sqd 200101_sqd 200102_sqd 200105_sqd 200106_sqd \
200107_sqd 200112_sqd 200201_sqd 200202_sqd 200206_sqd 200207_sqd 200212_sqd 200301_sqd \
200302_sqd 200306_sqd 200307_sqd 200311_sqd 200312_sqd 200401_sqd 200402_sqd 200405_sqd \
200406_sqd 200501_sqd 200502_sqd 200506_sqd 200601_sqd 200602_sqd 200606_sqd 200705_sqd \
200706_sqd 200707_sqd)

for the 1st csv file it will add a column using the first element of spname, 2nd csv file, 2nd element of spname and so on..
Bunch of thanks in advance.
Last edited by ida1215; 01-29-2012 at 09:25 PM..
# 2  
Old 01-30-2012
How are your csv files named? Does it follow a pattern?
# 3  
Old 01-30-2012
The csv filenames are patterned as below:

199906.csv 1999907.csv 199908.csv 199909.csv .. it corresponds to the first 6 characters of the each unique string within the array.

Thanks,Smilie
# 4  
Old 01-30-2012
So, basically you just want to append the respective filename with "_spd" to each row. Did I get it right?

--ahamed
# 5  
Old 01-30-2012
@ ahamed101. yeah that would be it.
# 6  
Old 01-30-2012
Try this...
Code:
awk '{if(filename!=FILENAME){file=filename=FILENAME;gsub(/.csv/,"_spd",file)}print file","$0>FILENAME".1"}' *.csv

--ahamed
This User Gave Thanks to ahamed101 For This Post:
ida1215
# 7  
Old 01-30-2012
From ahmed101's interpretation of your requirement:
Code:
for file in *.csv; do sed -i "s/^/`basename $file .csv`_sqd,/" $file; done

Going by your original idea:
Code:
#! /bin/bash

spname=(199906_sqd 199907_sqd 199908_sqd 199909_sqd 199910_sqd 200005_sqd 200006_qd 200007_sqd \
200008_sqd 200009_sqd 200010_sqd 200011_sqd 200101_sqd 200102_sqd 200105_sqd 200106_sqd \
200107_sqd 200112_sqd 200201_sqd 200202_sqd 200206_sqd 200207_sqd 200212_sqd 200301_sqd \
200302_sqd 200306_sqd 200307_sqd 200311_sqd 200312_sqd 200401_sqd 200402_sqd 200405_sqd \
200406_sqd 200501_sqd 200502_sqd 200506_sqd 200601_sqd 200602_sqd 200606_sqd 200705_sqd \
200706_sqd 200707_sqd)

for x in ${spname[@]}
do
    sed -i "s/^/$x,/" `echo $x | cut -d_ -f1`.csv
done

These 2 Users Gave Thanks to balajesuri For This Post:
ahamed101 ida1215
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split into multiple files by using Unique columns in a UNIX file

I have requirement to split below file (sample.csv) into multiple files by using the unique columns (first 3 are unique columns) sample.csv 123|22|56789|ABCDEF|12AB34|2019-07-10|2019-07-10|443.3400|1|1 123|12|5679|BCDEFG|34CD56|2019-07-10|2019-07-10|896.7200|1|2... (3 Replies)
Discussion started by: RVSP
3 Replies

2. Shell Programming and Scripting

Search string in multiple files and display column wise

I have 3 files. Each of those files have the same number of records, however certain records have different values. I would like to grep the field in ALL 3 files and display the output with only the differences in column wise and if possible line number File1 Name = Joe Age = 33... (3 Replies)
Discussion started by: sidnow
3 Replies

3. Shell Programming and Scripting

Count occurrence of column one unique value having unique second column value

Hello Team, I need your help on the following: My input file a.txt is as below: 3330690|373846|108471 3330690|373846|108471 0640829|459725|100001 0640829|459725|100001 3330690|373847|108471 Here row 1 and row 2 of column 1 are identical but corresponding column 2 value are... (4 Replies)
Discussion started by: angshuman
4 Replies

4. Shell Programming and Scripting

Unique entries in multiple files

Hello, I have a directory with a log files(many of them). Lines look like this: Sep 1 00:05:05 server9 pop3d-ssl: LOGIN, user=abc@example.com, ip=, port= Sep 1 00:05:05 server9 pop3d-ssl: LOGOUT, user=abc@example.com, ip=, port=, top=0, retr=0, rcvd=12, sent=46, time=0 Sep 1 00:05:05... (19 Replies)
Discussion started by: ramirez987
19 Replies

5. Shell Programming and Scripting

Merging two tables including multiple ocurrence of column identifiers and unique lines

I would like to merge two tables based on column 1: File 1: 1 today 1 green 2 tomorrow 3 red File 2: 1 a lot 1 sometimes 2 at work 2 at home 2 sometimes 3 new 4 a lot 5 sometimes 6 at work (4 Replies)
Discussion started by: BSP
4 Replies

6. Shell Programming and Scripting

Count Unique values from multiple lists of files

Looking for a little help here. I have 1000's of text files within a multiple folders. YYYY/ /MM /1000's Files Eg. 2014/01/1000 files 2014/02/1237 files 2014/03/1400 files There are folders for each year and each month, and within each monthly folder there are... (4 Replies)
Discussion started by: whegra
4 Replies

7. Shell Programming and Scripting

Compare columns of multiple files and print those unique string from File1 in an output file.

Hi, I have multiple files that each contain one column of strings: File1: 123abc 456def 789ghi File2: 123abc 456def 891jkl File3: 234mno 123abc 456def In total I have 25 of these type of file. (5 Replies)
Discussion started by: owwow14
5 Replies

8. Shell Programming and Scripting

Compare multiple files and print unique lines

Hi friends, I have multiple files. For now, let's say I have two of the following style cat 1.txt cat 2.txt output.txt Please note that my files are not sorted and in the output file I need another extra column that says the file from which it is coming. I have more than 100... (19 Replies)
Discussion started by: jacobs.smith
19 Replies

9. Shell Programming and Scripting

get part of file with unique & non-unique string

I have an archive file that holds a batch of statements. I would like to be able to extract a certain statement based on the unique customer # (ie. 123456). The end for each statement is noted by "ENDSTM". I can find the line number for the beginning of the statement section with sed. ... (5 Replies)
Discussion started by: andrewsc
5 Replies

10. Shell Programming and Scripting

Adding Multiple Lines to Multiple Files

Hello, I have three lines I need to add to hundreds of files. The files are all in the same format and contain the same information. I want to add the same information to the files. The new lines of information are similar. Here is an example of the three lines: line1: line2 =... (2 Replies)
Discussion started by: dayinthelife
2 Replies
Login or Register to Ask a Question