Sponsored Content
Top Forums Shell Programming and Scripting Need Header for all splitted files - awk Post 302426318 by mannefromdetroi on Tuesday 1st of June 2010 12:49:13 PM
Old 06-01-2010
Need Header for all splitted files - awk

Input file: i have a file and need to split into multiple files based on first column. i need the header for all the splitted files. I'm unable to get the header.
Code:
$ cat log.txt
id,mailtype,value
1252468812,yahoo,3.5
1252468812,hotmail,2.4
1252468819,yahoo,1.2
1252468812,msn,8.9
1252468923,gmail,12
1252468819,live,3.4
1252468929,yahoo,9.0
1252468929,msn,1.2

Required:
Split the above files based on the first field (i.e. lines with same first field should go to the same file)

The awk one liner:
Code:
$ awk -F "," '{close(f);f=$1}{print > f".txt"}' log.txt

Output:
Above file is splited into the following sub-files.
Code:
$ cat 1252468812.txt
1252468812,yahoo,3.5
1252468812,hotmail,2.4
1252468812,msn,8.9
 
$ cat 1252468819.txt
1252468819,yahoo,1.2
1252468819,live,3.4
 
$ cat 1252468923.txt
1252468923,gmail,12
 
$ cat 1252468929.txt
1252468929,yahoo,9.0
1252468929,msn,1.2


Last edited by Franklin52; 06-01-2010 at 02:32 PM.. Reason: Please use code tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to restore splitted DD file

I backup a file to tape by using "dd", since the size is too large it took two tape for the backup, the question is how to restore the said file? The "dd" doesn't ask me insert next tape while restoration. (3 Replies)
Discussion started by: coolmans
3 Replies

2. Shell Programming and Scripting

Merging words splitted into characters with awk

I have an OCR output with some words splitted into single characters separated by blank spaces, and I want the same text with these words written correctly. Example: This is a text w i t h some s p l i t e d W o r d s . The regular expression for matching splitted words could be something... (5 Replies)
Discussion started by: dokamo
5 Replies

3. UNIX for Dummies Questions & Answers

Merge all csv files in one folder considering only 1 header row and ignoring header of all others

Friends, I need help with the following in UNIX. Merge all csv files in one folder considering only 1 header row and ignoring header of all other files. FYI - All files are in same format and contains same headers. Thank you (4 Replies)
Discussion started by: Shiny_Roy
4 Replies

4. Shell Programming and Scripting

splitting newfile.txt file and executing each splitted files

split -l $split_count newfile.txt for i in $split_files* do if test -s $workingdir/$split_files* then ./<$i.out> fi done ... (4 Replies)
Discussion started by: sanjay mn
4 Replies

5. Shell Programming and Scripting

Adding header to sub files after splitting the main file using AWK

Hi Folks, I have a file like: mainfile.txt: ------------- file1 abc def xyz file1 aaa pqr xyz file2 lmn ghi xyz file2 bbb tuv xyz I need output having two files file1 and file2. file1: ------ Name State Country abc def xyz aaa pqr xyz file2: (3 Replies)
Discussion started by: tanmay.gemini
3 Replies

6. Shell Programming and Scripting

Script to multi-transfer splitted files via scp

Hey :3 I am moving some stuff between different servers. I do it like this: scp -r -P 22 -i ~/new.ppk /var/www/bigfile.tar.gz user@123.123.123.123:/var/www/bigfile.tar.gz Lets say, this file is 50 GiB. I would like to know, if its possible to split the file in different parts,... (2 Replies)
Discussion started by: Keenora
2 Replies

7. Shell Programming and Scripting

Combine splitted low & high byte files into one file

Hi all, i have a binary file splitted into 2 chunks, first part with all high bytes and the second part with all low bytes. I need to combine the two chunks into one binary file like (eg. exactly the reverse of the splitting method solved in the thread # 130940) Hi bytes file content:... (7 Replies)
Discussion started by: mzs
7 Replies

8. Shell Programming and Scripting

Assistance with an awk code to split files but keep the header

---------- Post updated at 11:48 AM ---------- Previous update was at 11:46 AM ---------- Hello all I have an awk code that successfully creates separate text files based on the first six letters of the second field. What it doesn't do is preserve the header into each resulting file. ... (6 Replies)
Discussion started by: colecandoo
6 Replies

9. AIX

Mail subject splitted

Hi I have a script that is sending mail and it was working fine. But the last time it run, it gave some errors. It look like the subject was split into many receiver. I am on AIX 6.1. #!/bin/ksh # HOSTNAME=`which hostname` MACHINE=`${HOSTNAME}` # E-mail to inform hci of a stop echo "###... (9 Replies)
Discussion started by: jacquesT
9 Replies

10. UNIX for Beginners Questions & Answers

Append header with awk

I have to append the header (ie "START OF LINE") to a file only if there is data in it. Below command is showing header though there is no data in a file, can you suggest how to correct.. awk '{NR!=0} BEGIN{print "START OF LINE"}; {print}' file (3 Replies)
Discussion started by: JSKOBS
3 Replies
MINMAX(l)																 MINMAX(l)

NAME
minmax - Find extreme values in data tables SYNOPSIS
minmax [ files] [ -C ] [ -D ] [ -H[nrec] ] [ -Idx[/dy] ] [ -L ] [ -M[flag] ] [ -: ] [ -bi[s][n] ] DESCRIPTION
minmax reads its standard input [or from files] and finds the extreme values in each of the columns. It recognizes NaNs and will print warnings if the number of columns vary from record to record. As an option, minmax will find the extent of the first two columns rounded up and down to the nearest multiple of dx/dy. This output will be in the form -Rw/e/s/n which can be used directly in the command line for other programs, or simply in column form. xyzfile ASCII [or binary, see -b] file(s) holding a fixed number of data columns. OPTIONS
-C Report the min/max values per column in separate columns [Default uses <min/max> format] -D Sets longitude discontinuity to the Dateline (-180/+180) [Default is Greenwich (0-360)]. Requires -L. -H Input file(s) has Header record(s). Number of header records can be changed by editing your .gmtdefaults file. If used, GMT default is 1 header record. -I Report the min/max of the first two columns to the nearest multiple of dx and dy, and output this in the form -Rw/e/s/n (unless -C is set). -L Indicates that the x column contains longitudes, which may be periodic in 360 degrees [Default assumes no periodicity]. -M Multiple segment file(s). Segments are separated by a special record. For ASCII files the first character must be flag [Default is '>']. For binary files all fields must be NaN. -: Toggles between (longitude,latitude) and (latitude,longitude) input/output. [Default is (longitude,latitude)]. Applies to geo- graphic coordinates only. Only works when -I is selected. -bi Selects binary input. Append s for single precision [Default is double]. Append n for the number of columns in the binary file(s). [Default is 2 input columns]. EXAMPLES
To find the extreme values in the file ship_gravity.xygd, try minmax ship_gravity.xygd Output should look like ship_gravity.xygd: N = 6992 <326.125/334.684> <-28.0711/-8.6837> <-47.7/177.6> <0.6/3544.9> To find the extreme values in the file track.xy to the nearest 5 units and use this region to draw a line using psxy, try psxy `minmax -I5 track.xy` track.xy -Jx1 -B5 -P > track.ps To find the min and max values for each column, but rounded to integers, try minmax junkfile -C -I1 SEE ALSO
gmt(1gmt) 1 Jan 2004 MINMAX(l)
All times are GMT -4. The time now is 11:01 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy