Writing awk script to read csv files and split them


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Writing awk script to read csv files and split them
# 1  
Old 09-21-2010
Writing awk script to read csv files and split them

Hi

Here is my script that calls my awk script
Code:
#!/bin/bash
set -x
dir="/var/local/dsx/csv"
testfile="$testfile"
while getopts " f: " option
do
 case $option in
  
  f ) testfile="$OPTARG";;
 esac;
done

./scriptFile --testfile=$testfile >> $dir/$testfile.csv

It calls my awk script:
Code:
#!/bin/bash
awk -F, '
 $1 == last1 && $2 == last2 {
  printf $0"\n" >>$2
 }
 $1 != last1 || $2 != last2 {
  printf $0"\n" >>$2
 }
  {
  last1=$1
  last2=$2 
 }
 ' 
 $1

Here is my csv

Code:
o,e,r
1,2,3
1,2,3
1,2,3
4,5,6
4,5,6
4,5,6
4,5,6
4,5,6
4,5,6

File1 should look like this:
Code:
1,2,3
1,2,3
1,2,3

File2
Code:
4,5,6
4,5,6
4,5,6
4,5,6
4,5,6
4,5,6

And so forth

I need help to make the awk script work

Last edited by pludi; 09-21-2010 at 10:00 AM..
# 2  
Old 09-21-2010
See your first thread.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk script to perform an action similar to vlookup between two csv files in UNIX

Hi, I am new to awk/unix and am trying to put together an awk script to perform an action similar to vlookup between the two csv files. Here are the contents of the two files: File 1: Date,ParentID,Number,Area,Volume,Dimensions 2014-01-01,ABC,247,83430.33,857.84,8110.76... (9 Replies)
Discussion started by: Prit Siv
9 Replies

2. Shell Programming and Scripting

Split a .csv File into Multiple Files

Hi guys, I have a requirement where i need to split a .csv file into multiple files. Say for example i have data.csv file and i have splitted that into multiple files based on some conditions i.e first file should have 100, last file 50 and other files 1000 each. Am passing the values in... (2 Replies)
Discussion started by: azherkn3
2 Replies

3. Shell Programming and Scripting

awk script to split file into multiple files based on many columns

So I have a space delimited file that I'd like to split into multiple files based on multiple column values. This is what my data looks like 1bc9A02 1 10 1000 FTDLNLVQALRQFLWSFRLPGEAQKIDRMMEAFAQRYCQCNNGVFQSTDTCYVLSFAIIMLNTSLHNPNVKDKPTVERFIAMNRGINDGGDLPEELLRNLYESIKNEPFKIPELEHHHHHH 1ku1A02 1 10... (9 Replies)
Discussion started by: viored
9 Replies

4. Shell Programming and Scripting

awk read column csv and search in other csv

hi, someone to know how can i read a specific column of csv file and search the value in other csv columns if exist the value in the second csv copy entire row with all field in a new csv file. i suppose that its possible using awk but i m not expertise thanks in advance (8 Replies)
Discussion started by: giankan
8 Replies

5. Shell Programming and Scripting

split a csv file into specified number of files (not lines)

hi, i really need it ...it's not simple to explain but as it's part of a crontab i can't split the file manually...and the file can change every day so the lines are not a good base. example: how to split 1 csv file in 15 files? thank you very much regards :b: (4 Replies)
Discussion started by: 7stars
4 Replies

6. Shell Programming and Scripting

Split line to multiple files Awk/Sed/Shell Script help

Hi, I need help to split lines from a file into multiple files. my input look like this: 13 23 45 45 6 7 33 44 55 66 7 13 34 5 6 7 87 45 7 8 8 9 13 44 55 66 77 8 44 66 88 99 6 I want to split every 3 lines from this file to be written to individual files. (3 Replies)
Discussion started by: saint2006
3 Replies

7. UNIX for Dummies Questions & Answers

Split words from 2 files and printing in csv

My task requires me to add a column to my existing csv file.This column will be populated with data that is dependant on data in another column.The script will go through the data in column A and create new cells in column B that will be dependent on column A data. Thanks (1 Reply)
Discussion started by: Adan Daroski
1 Replies

8. UNIX for Dummies Questions & Answers

Split words from 2 files and printing in csv

Hello Everyone, (1 Reply)
Discussion started by: thankful123
1 Replies

9. Shell Programming and Scripting

Sed or awk script to remove text / or perform calculations from large CSV files

I have a large CSV files (e.g. 2 million records) and am hoping to do one of two things. I have been trying to use awk and sed but am a newbie and can't figure out how to get it to work. Any help you could offer would be greatly appreciated - I'm stuck trying to remove the colon and wildcards in... (6 Replies)
Discussion started by: metronomadic
6 Replies

10. Shell Programming and Scripting

help writing script to read files names

Hi there, I am trying to do somehting similar, but on a wider scale. I am trying to write a script that would open the home directory, open the first (of 650) user's folder open the ?mail directory, which every user has Then I need the script to read each of the files and folder names with... (2 Replies)
Discussion started by: technett
2 Replies
Login or Register to Ask a Question