Sponsored Content
Top Forums Shell Programming and Scripting Merge text files while combining the multiple header/trailer records into one each. Post 302259235 by jim mcnamara on Monday 17th of November 2008 02:03:09 PM
Old 11-17-2008
assuming this: 01E000036841 is an employee id and the files are named <something>.dat
Code:
ls *.dat | read header dummy
# save copies of header
head -1 $header > tmp

awk '{ if (index($0, "HEADER") > 0 || index($0, "TRAILER") >0 ) {last= $0; continue}
       arr[$0]++; print $0   }
       END { for (i in arr) 
             {
               empcnt++ 
               lc+=arr[i]
             } 
             print empcnt, lc > "cntfile" }  ' *.dat >> tmp
awk ' { rec=sprintf("%08d%08d", $1 $2)}
         END { printf("BATCH TRAILER %s%s\n", rec, substr(last, length(last)-7) } ' cntfile >> tmp
mv tmp employee.dat

This also assumes the last eight characters of BATCH TRAILER are all the same.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Count No of Records in File without counting Header and Trailer Records

I have a flat file and need to count no of records in the file less the header and the trailer record. I would appreciate any and all asistance Thanks Hadi Lalani (2 Replies)
Discussion started by: guiguy
2 Replies

2. UNIX for Dummies Questions & Answers

Copy all the files with time stamp and remove header,trailer from file

All, I am new to unix and i have the following requirement. I have file(s) landing into input directory with timestamp, first i want to copy all these files into seperate directory then i want to rename these files without timestamp and also remove header,trailer from that file.. Could... (35 Replies)
Discussion started by: ksrams
35 Replies

3. Shell Programming and Scripting

improve performance - replace $\| with $#@ and remove header and trailer records

Hi All, In my file i need to remove header and trailer records which comes in 1st line and last line respectively. After that i need to replace '$\|' with '$#@'. I am using sed command for this and its taking lot of time. Is there any other command which can be used to improve performance? ... (1 Reply)
Discussion started by: HemaV
1 Replies

4. Shell Programming and Scripting

Adding Header and Trailer records to a appended file

How can we a shell script and pass date parameters .I have 3 files comming from Datastage with |" delimited I need append 3 files as above: File1: P0000|"47416954|"AU|"000|"INS|"0000|"|"20060601|"99991231|"|"|"|"|"01 File 2:... (2 Replies)
Discussion started by: e1994264
2 Replies

5. UNIX for Dummies Questions & Answers

Merge all csv files in one folder considering only 1 header row and ignoring header of all others

Friends, I need help with the following in UNIX. Merge all csv files in one folder considering only 1 header row and ignoring header of all other files. FYI - All files are in same format and contains same headers. Thank you (4 Replies)
Discussion started by: Shiny_Roy
4 Replies

6. UNIX for Dummies Questions & Answers

Need help combining txt files w/ multiple lines into csv single cell - also need data merge

:confused:Hello -- i just joined the forums. I am a complete noob -- only about 1 week into learning how to program anything... and starting with linux. I am working in Linux terminal. I have a folder with a bunch of txt files. Each file has several lines of html code. I want to combine... (2 Replies)
Discussion started by: jetsetter
2 Replies

7. Shell Programming and Scripting

Merge the multiple text files into one file

Hi All, I am trying to merge all the text files into one file using below snippet cat /home/Temp/Test/Log/*.txt >> all.txt But it seems it is not working. I have multiple files like Output_ServerName1.txt, Output_ServreName2.txt I want to merge each file into one single file and... (6 Replies)
Discussion started by: sharsour
6 Replies

8. Shell Programming and Scripting

Need to merge multiple text files vertically and place comma between fields

Hello expert friends, I'm writing a script to capture stats using sar and stuck up at report generation. I have around 10 files in a directory and need to merge them all vertically based on the time value of first column (output file should have only one time value) and insert comma after... (6 Replies)
Discussion started by: prvnrk
6 Replies

9. Shell Programming and Scripting

Merge multiple files with common header

Hi all, Say i have multiple files x1 x2 x3 x4, all with common header (date, time, year, age),, How can I merge them to one singe file "X" in shell scripting Thanks for your suggestions. (2 Replies)
Discussion started by: msarguru
2 Replies

10. UNIX for Beginners Questions & Answers

Append pipe | at the end of all the rows except header n trailer for all the files under a directory

Hi Experts Need help... I am looking for a Unix script to append pipe | at the end of all the rows (except header and trailer)in all the files placed under the directory /interfaces/Temp e.g. Header row1 row2 row3 Trailer The script should read all the files under... (3 Replies)
Discussion started by: phani333
3 Replies
kst(1)								       kst2								    kst(1)

NAME
kst -- plots scientific data SYNOPSIS
kst kst [ -F DATASOURCE ] [ -f STARTFRAME ] [ -n NUMFRAMES ] [ -s FRAMESPERSAMPLE [ -a ] ] kstfile kst datasource [ -f STARTFRAME ] [ -n NUMFRAMES ] [ -s FRAMESPERSAMPLE [ -a ] ] [ -P PLOTNAME ] [ -A] [ -d ] [ -l ] [ -b ] [ -x FIELD ] [ -e FIELD ] [ -r RATE ] [ -y FIELD ] [ -p FIELD ] [ -h FIELD ] [ -z FIELD ] [ --png filename ] [ --print filename [ --landscape | --portrait ] [ --Letter | --A4 ] ] DESCRIPTION
kst is a graphical data plotting tool, which includes the ability to plot live data (contiuously update the plot as new data becomes avail- able). kst may be run entirely from within its GUI, in which case no command line arguments are needed. kst has a powerful wizard for easily set- ting up new plots. In the second invocation, kst loads pre-saved plot specifications from the file kstfile and optionally may override some of the settings in that file. In the third invocation, kst displays plots of the data as directed in the default manner. Additional datasources may be specified on the command line with options for that datasource following it. Command line options generally apply to all subsequent options unless later overridden. OPTIONS
-A plot all curves in separate plots -a average frames (simple mean) when plotting with -s. --A4 use A4 sized paper for printing. Requires --print. -b plot as bargraphs. -d use points when plotting curves. -e FIELD use FIELD as the magnitude of error bars applied to the next plot. -F datasource override the datasource indicated in the kstfile and instead use datasource. -f STARTFRAME begin plotting at STARTFRAME frames into the data. Set to -1 to start at NUMFRAMES from the end of the data. -h FIELD plot FIELD as a histogram. -l use lines when plotting curves. --landscape use landscape orientation for printing. Requires --print. --Letter use US Letter sized paper for printing. Requires --print. -n NUMFRAMES plot at most NUMFRAMES frames of data. Set to -1 to indicate all of the data. -P PLOTNAME plot all plots in the plot named PLOTNAME. -p FIELD plot FIELD as a power spectrum plot. --png filename render to a portable network graphics (png) image with file name filename and quit. --portrait use portrait orientation for printing. Requires --print. --print filename print the plot to file filename and quit. -s FRAMESPERSAMPLE plot every FRAMESPERSAMPLE frames when plotting. -x FIELD use FIELD as the X-axis vector. -y FIELD plot FIELD as an XY plot. -z FIELD plot FIELD as an image. EXAMPLES
Plot all data in column 2 from data.dat. kst data.dat -y 2 Same as above, except only read 20 lines, starting at line 10. kst data.dat -f 10 -n 20 -y 2 also read col 1. One plot per curve. kst data.dat -f 10 -n 20 -y 1 -y 2 Read col 1 from data2.dat and col 1 from data.dat kst data.dat -f 10 -n 20 -y 2 data2.dat -y 1 Same as above, except read 40 lines starting at 30 in data2.dat kst data.dat -f 10 -n 20 -y 2 data2.dat -f 30 -n 40 -y 1 Specify the X vector and error bars: Plot x = col 1 and Y = col 2 and error flags = col 3 from data.dat kst data.dat -x 1 -e 3 -y 2 Get the X vector from data1.dat, and the Y vector from data2.dat. kst data1.dat -x 1 data2.dat -y 1 Placement: Plot column 2 and column 3 in plot P1 and column 4 in plot P2 kst data.dat -P P1 -y 2 -y 3 -P P2 -y 4 BUGS
Please report bugs to either the kst mailing list at kst@kde.org or to the kde bugzilla under the kst product at http://bugs.kde.org/. If you can run kst, you can easily report bugs by selecting Bug Report Wizard from the Help menu. AUTHOR
Matthew Truch <matt@truch.net> LICENSE
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the `COPYING.DOC' file as part of the kst distribution. Version 2.0.3 22 February 2011 kst(1)
All times are GMT -4. The time now is 06:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy