Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Writing awk script to read csv files and split them Post 302455292 by ladyAnne on Tuesday 21st of September 2010 08:20:54 AM
Old 09-21-2010
Writing awk script to read csv files and split them

Hi

Here is my script that calls my awk script
Code:
#!/bin/bash
set -x
dir="/var/local/dsx/csv"
testfile="$testfile"
while getopts " f: " option
do
 case $option in
  
  f ) testfile="$OPTARG";;
 esac;
done

./scriptFile --testfile=$testfile >> $dir/$testfile.csv

It calls my awk script:
Code:
#!/bin/bash
awk -F, '
 $1 == last1 && $2 == last2 {
  printf $0"\n" >>$2
 }
 $1 != last1 || $2 != last2 {
  printf $0"\n" >>$2
 }
  {
  last1=$1
  last2=$2 
 }
 ' 
 $1

Here is my csv

Code:
o,e,r
1,2,3
1,2,3
1,2,3
4,5,6
4,5,6
4,5,6
4,5,6
4,5,6
4,5,6

File1 should look like this:
Code:
1,2,3
1,2,3
1,2,3

File2
Code:
4,5,6
4,5,6
4,5,6
4,5,6
4,5,6
4,5,6

And so forth

I need help to make the awk script work

Last edited by pludi; 09-21-2010 at 10:00 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

help writing script to read files names

Hi there, I am trying to do somehting similar, but on a wider scale. I am trying to write a script that would open the home directory, open the first (of 650) user's folder open the ?mail directory, which every user has Then I need the script to read each of the files and folder names with... (2 Replies)
Discussion started by: technett
2 Replies

2. Shell Programming and Scripting

Sed or awk script to remove text / or perform calculations from large CSV files

I have a large CSV files (e.g. 2 million records) and am hoping to do one of two things. I have been trying to use awk and sed but am a newbie and can't figure out how to get it to work. Any help you could offer would be greatly appreciated - I'm stuck trying to remove the colon and wildcards in... (6 Replies)
Discussion started by: metronomadic
6 Replies

3. UNIX for Dummies Questions & Answers

Split words from 2 files and printing in csv

Hello Everyone, (1 Reply)
Discussion started by: thankful123
1 Replies

4. UNIX for Dummies Questions & Answers

Split words from 2 files and printing in csv

My task requires me to add a column to my existing csv file.This column will be populated with data that is dependant on data in another column.The script will go through the data in column A and create new cells in column B that will be dependent on column A data. Thanks (1 Reply)
Discussion started by: Adan Daroski
1 Replies

5. Shell Programming and Scripting

Split line to multiple files Awk/Sed/Shell Script help

Hi, I need help to split lines from a file into multiple files. my input look like this: 13 23 45 45 6 7 33 44 55 66 7 13 34 5 6 7 87 45 7 8 8 9 13 44 55 66 77 8 44 66 88 99 6 I want to split every 3 lines from this file to be written to individual files. (3 Replies)
Discussion started by: saint2006
3 Replies

6. Shell Programming and Scripting

split a csv file into specified number of files (not lines)

hi, i really need it ...it's not simple to explain but as it's part of a crontab i can't split the file manually...and the file can change every day so the lines are not a good base. example: how to split 1 csv file in 15 files? thank you very much regards :b: (4 Replies)
Discussion started by: 7stars
4 Replies

7. Shell Programming and Scripting

awk read column csv and search in other csv

hi, someone to know how can i read a specific column of csv file and search the value in other csv columns if exist the value in the second csv copy entire row with all field in a new csv file. i suppose that its possible using awk but i m not expertise thanks in advance (8 Replies)
Discussion started by: giankan
8 Replies

8. Shell Programming and Scripting

awk script to split file into multiple files based on many columns

So I have a space delimited file that I'd like to split into multiple files based on multiple column values. This is what my data looks like 1bc9A02 1 10 1000 FTDLNLVQALRQFLWSFRLPGEAQKIDRMMEAFAQRYCQCNNGVFQSTDTCYVLSFAIIMLNTSLHNPNVKDKPTVERFIAMNRGINDGGDLPEELLRNLYESIKNEPFKIPELEHHHHHH 1ku1A02 1 10... (9 Replies)
Discussion started by: viored
9 Replies

9. Shell Programming and Scripting

Split a .csv File into Multiple Files

Hi guys, I have a requirement where i need to split a .csv file into multiple files. Say for example i have data.csv file and i have splitted that into multiple files based on some conditions i.e first file should have 100, last file 50 and other files 1000 each. Am passing the values in... (2 Replies)
Discussion started by: azherkn3
2 Replies

10. Shell Programming and Scripting

awk script to perform an action similar to vlookup between two csv files in UNIX

Hi, I am new to awk/unix and am trying to put together an awk script to perform an action similar to vlookup between the two csv files. Here are the contents of the two files: File 1: Date,ParentID,Number,Area,Volume,Dimensions 2014-01-01,ABC,247,83430.33,857.84,8110.76... (9 Replies)
Discussion started by: Prit Siv
9 Replies
mkcdsl(8)						      System Manager's Manual							 mkcdsl(8)

NAME
mkcdsl - Make a context-dependent symbolic link (CDSL) SYNOPSIS
/usr/sbin/mkcdsl [-bfnqv] [-i | -c | -a] [sourcename] [targetname] OPTIONS
All members. Copy the existing file or directory to a member-specific area on all members. Boot partition. In a cluster, the CDSL is placed on the boot partition. On a standalone system, an entry in added to the CDSL inventory file, and the CDSL will be created on the boot partition when the system is converted to a cluster by running clu_create. Copy the existing file or directory to a member-specific area on this member only. Force the overwriting of the existing CDSL or member-specific file or directory. When the force option is used with a copy option, mkcdsl will overwrite an existing member-specific file or directory. Without the force (-f) option, mkcdsl issues an error or message whenever the physical path of the target differs from the specified targetname (for example, when targetname resolution traverses a symbolic link), or when the source for a specified copy option cannot be found. Unless the -f option is specified, mkcdsl will exit when it encounters a situation that would generate an error message. The mkcdsl command issues a warning message if the specified sourcename differs from the calculated sourcename. However, you do not need the -f option to stop mkcdsl from exiting when it encounters a situation that generates a warning message. Inventory only. Check the files and update the inventory: If targetname is a CDSL, add a new entry or update the existing entry in /var/adm/cdsl_admin.inv. If targetname is not a CDSL or does not exist, remove its entry from /var/adm/cdsl_admin.inv. No execute mode. Display what would be done but do not create or change anything. Quiet. Stdout and stderr are redirected to /dev/null. Verbose. The pathname of the member-specific file to be linked. By default, each CDSL created by mkcdsl has a sourcename that points to the default member-specific area in the file system where the targetname resolves. For example, /usr/cluster/mem- bers/{memb}/ is the default member-specific area for the /usr file system. When issued without an explicit sourcename, mkcdsl creates a sourcename that points to this default member-specific area using a relative pathname. In most cases, this default behavior is satisfac- tory. It creates a relative link that resolves properly regardless of where the file system is mounted. If you specify a sourcename, mkcdsl uses this value as the pathname to the member-specific file or directory. The sourcename must include the member-context path compo- nent {memb}. The pathname of the new CDSL entry to be created. If targetname exists, the -f, -c or -a options are required. If the reso- lution of the targetname traverses any symbolic links or file system mount points, the default value of sourcename is adjusted such that the member-specific files are placed in the same file system to which targetname resolves. DESCRIPTION
A context-dependent symbolic link (CDSL) is a special form of a symbolic link, described in ln(1), whose target pathname includes a vari- able ({memb}). The value of {memb} is determined during pathname resolution, making it possible to maintain member-specific configuration and data files within the clusterwide namespace. The mkcdsl command creates a CDSL, effectively making the specified targetname file or directory member-specific. The command simplifies the creation of a CDSL by automatically calculating a default sourcename, which resolves to the default member-specific area in the file system where the CDSL is created. When mkcdsl creates a CDSL, it adds an inventory entry to a clusterwide CDSL inventory file, /var/adm/cdsl_admin.inv. This entry makes it possible for the cdslinvchk command to check CDSLs created by mkcdsl. The mkcdsl -i option updates the inventory file to reflect the cur- rent state of the CDSL by adding, modifying, or removing entries from the CDSL inventory file. In a cluster, when directed to put a file in a member-specific directory (-c or -a), mkcdsl populates both the member-specific directory or directories and the member0 directory. Although you can run mkcdsl from the command line, the mkcdsl command is designed to be non-interactive; it can be called from scripts without requiring user input to complete a task. EXAMPLES
To create a CDSL, /usr/testfile that points to cluster/members/{memb}/testfile: # mkcdsl /usr/testfile # ls -l /usr/testfile ... /usr/testfile -> cluster/members/{memb}/testfile In the previous example, /usr/testfile must not exist. The CDSL is created in /usr and no files are created in any member's /usr/clus- ter/members/{memb} directory. To copy the existing file or directory /usr/testfile1 to this member's /usr/cluster/members/membern directory and replace it with a CDSL that points to cluster/members/{memb}/testfile1: # mkcdsl -c /usr/testfile1 Although the CDSL references all members, only this member has a copy of the original /usr/testfile1 file. To copy the existing file or directory /usr/testfile2 to all members' /usr/cluster/members/{memb} directories and replace it with a CDSL that points to cluster/members/{memb}/testfile2: # mkcdsl -a /usr/testfile2 All members have a copy of the original /usr/testfile2 file. To create a CDSL and explicitly specify the sourcename: # mkcdsl /usr/share/cluster/members/{memb}/testfile3 /usr/share/doclib/testfile3 *** Warning *** The passed source name, '/usr/share/cluster/members/{memb}/testfile3' differs from the calculated source name: cluster/mem- bers/{memb}/testfile3 Using passed source name: '/usr/share/cluster/members/{memb}/testfile3' Because the specified sourcename differs from mkcdsl's calculated default value, the command issues a warning message. Note that mkcdsl makes the CDSL. The mkcdsl command does not exit on a warning message. It will exit for an *** Error *** message unless the -f option is specified. To add an inventory record for an existing CDSL to the /var/adm/cdsl_admin.inv file: # mkcdsl -i /etc/testfile4 To remove a CDSL and then remove its entry from the /var/adm/cdsl_admin.inv file: # rm /etc/testfile4 # mkcdsl -i /etc/testfile4 Note that removing the CDSL and its entry does not remove any files referenced by the CDSL. EXIT STATUS
The exit values for mkcdsl are: Success. Failure. FILES
Specifies the command path. CDSL administrative inventory file. This file will not exist on all systems. SEE ALSO
Commands: ln(1), cdslinvchk(8) Files: local(4) Miscellaneous: hier(5) mkcdsl(8)
All times are GMT -4. The time now is 11:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy