Sponsored Content
Top Forums Shell Programming and Scripting Extract header data from one file and combine it with data from another file Post 302738869 by Don Cragun on Monday 3rd of December 2012 04:00:55 AM
Old 12-03-2012
Try saving the following script in a file (I used the name runner while testing it, but choose anything you like). Make it executable using chmod +x file_name and run it using ./runner header.txt main.txt to test it with the sample files you provided. The output this awk script produces match the output you requested (as long as you remove the trailing empty line at the end of main.txt) except that there are no spaces before tabs in the header line:
Code:
#!/bin/ksh
# Usage: runner hdr main [hdr main]...
if [ $# -lt 2 ] || [ $(($# % 2)) -ne 0 ]
then    printf "%s: Odd number of operands or less than two operands:\n" $0 >&2
        printf "Usage: %s hdr main [hdr main]...\n" "$0" >&2
        exit 1
fi
while [ $# -ge 2 ]
do      hf="$1"
        mf="$2"
        shift 2
        awk 'BEGIN{
                m["Jan"] = 1; m["Feb"] = 2;  m["Mar"] = 3;  m["Apr"] = 4
                m["May"] = 5; m["Jun"] = 6;  m["Jul"] = 7;  m["Aug"] = 8
                m["Sep"] = 9; m["Oct"] = 10; m["Nov"] = 11; m["Dec"] = 12
                FS = "[:=] "
                printf("System UpLoad\tTime\tSHIP\tCRUISE\tSTATION\tLAT\t%s",
                        "LON\tNATURE_OF_PR\tPROJ_NO\tINSTITUTE_CD\tST_DEPTH\n")
        }
        FNR>1 && FNR!=NR{
                printf("%s\t%s\t%s\t%s\t%s\n", hdr, $1, $2, $3, $4)
                next
        }       
        FNR==NR && /System UpLoad/{
                split($2, a, "  *")
                dt=sprintf("%02d/%02d/%02d", a[2], m[a[1]], a[3] % 100)
                tm=a[4]
                next
        }
        FNR==NR && /SHIP/{sh = $2;next}
        FNR==NR && /CRUISE/{cr = $2;next}
        FNR==NR && /STATION/{st = $2;next}
        FNR==NR && /LAT/{la = $2;next}
        FNR==NR && /LONG/{lo = $2;next}
        FNR==1 && NR>1{
                FS = "\t\t*"
                hdr = sprintf("%s\t%s\t%s\t%s\t%s\t%s\t%s",
                        dt, tm, sh, cr, st, la, lo)
                next;
        }' "$hf" "$mf"
done

You can invoke this script with any even number of files >=2 where the 1st file in each pair is in the format of header.txt and the 2nd file in each pair is in the format of main.txt.
This User Gave Thanks to Don Cragun For This Post:
 

10 More Discussions You Might Find Interesting

1. Programming

to find header in Mp3 file and retrieve data

hi all, In an mp3 file , data is arranged in sequence of header and data ,how to retrieve data between two headers. Is the data between two headers fixed? because as per theory it says 1152 samples will be there , but dont knw how many bits one sample correspond to? it would help if any c... (2 Replies)
Discussion started by: shashi
2 Replies

2. Shell Programming and Scripting

insert a header in a huge data file without using an intermediate file

I have a file with data extracted, and need to insert a header with a constant string, say: H|PayerDataExtract if i use sed, i have to redirect the output to a seperate file like sed ' sed commands' ExtractDataFile.dat > ExtractDataFileWithHeader.dat the same is true for awk and... (10 Replies)
Discussion started by: deepaktanna
10 Replies

3. Shell Programming and Scripting

Extract data based on match against one column data from a long list data

My input file: data_5 Ali 422 2.00E-45 102/253 140/253 24 data_3 Abu 202 60.00E-45 12/23 140/23 28 data_1 Ahmad 256 7.00E-45 120/235 140/235 22 data_4 Aman 365 8.00E-45 15/65 140/65 20 data_10 Jones 869 9.00E-45 65/253 140/253 18... (12 Replies)
Discussion started by: patrick87
12 Replies

4. Shell Programming and Scripting

Extract specific content from data and rename its header problem asking

Input file 1: >pattern_5 GAATTCGTTCATGTAGGTTGASDASFGDSGRTYRYGHDGSDFGSDGGDSGSDGSDFGSDF ATTTAATTATGATTCATACGTCATATGTTATTATTCAATCGTATAAAATTATGTGACCTT SDFSDGSDFKSDAFLKJASLFJASKLFSJAKJFHASJKFHASJKFHASJKFHSJAKFHAW >pattern_1 AAGTCTTAAGATATCACCGTCGATTAGGTTTATACAGCTTTTGTGTTATTTAAATTTGAC... (10 Replies)
Discussion started by: patrick87
10 Replies

5. Shell Programming and Scripting

Extract data from an XML file & write into a CSV file

Hi All, I am having an XML tag like: <detail sim_ser_no_1="898407109001000090" imsi_1="452070001000090"> <security>ADM1=????</security> <security>PIN1=????</security> <security>PIN2=????</security> ... (2 Replies)
Discussion started by: ss_ss
2 Replies

6. Shell Programming and Scripting

get data from files combine them to a file

hi people; this is my file1.txt:192.168.1.1 192.168.1.2 192.168.1.3 192.168.1.4 ... this is my file2.txt:portnames usernames maxusercap ... i want to write to file3.txt:l ./getports 192.168.1.1 'get all;l+;get . portnames;l-' l ./getports 192.168.1.1 'get all;l+;get . usernames;l-'... (4 Replies)
Discussion started by: gc_sw
4 Replies

7. Shell Programming and Scripting

Extract data from XML file and write in CSV file

Hi friend i have input as following XML file <?xml version="1.0"?> <Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.054.001.02"> <BkToCstmrDbtCdtNtfctn> <GrpHdr><MsgId>LBP-RDJ-TE000000-130042430010001001</MsgId><CreDtTm>2013-01-04T03:21:30</CreDtTm></GrpHdr>... (3 Replies)
Discussion started by: mohan sharma
3 Replies

8. Shell Programming and Scripting

How to combine data from 2 file ? How to print ?

Dear friends, I am just trying to print data from 2 file,namely file_1.txt and file_1.dat (specific column of ROW 1) file_1.txt 12 13 14 15 99 AMC 69 36 89 12 13 14 15 99 AMC 69 84 -12 12 13 14 ... (6 Replies)
Discussion started by: nex_asp
6 Replies

9. Shell Programming and Scripting

Combine data out of 3 files into one new file

Hi, How can I combine the data of of three files into one new file? I try to give as much informations as possible. The three existing files are called file1 file2 and file3 the new file should named output_combined. The size of the files will be around 900 words/lines each .. but always... (5 Replies)
Discussion started by: MyMemberName
5 Replies

10. Shell Programming and Scripting

Bash to verify and validate file header and data type

The below bash is a file validation check executed that will verify the correct header count of 10 and the correct data type in each field of the tab-delimited file. The key has the data type of each field in it. My real data has 58 headers in it but only the header and next row need to be... (6 Replies)
Discussion started by: cmccabe
6 Replies
PX_SET_INPUTENCODING(3) 				     Library Functions Manual					   PX_SET_INPUTENCODING(3)

NAME
PX_set_inputencoding -- Set encoding for output SYNOPSIS
#include <paradox.h> int PX_set_inputencoding(pxdoc_t *pxdoc, const char *encoding) DESCRIPTION
Sets the input encoding for all alpha fields. It is used when data is stored into a record. The encoding must be a name as it is described by the man and info pages of recode or iconv whatever is used. A common value is `latin1' on Unix systems. If you do not set the encoding the function PX_get_data_alpha(3) will return field data in the original encoding otherwise it will recode the field data. This function is a short cut form PX_set_parameter(pxdoc, "inputencoding", encoding) and is deprecated. RETURN VALUE
Returns -2 if recoding is not supported and a value < 0 in case of an error, otherwise 0. SEE ALSO
PX_set_data_alpha(3), PX_set_targetencoding(3), PX_set_parameter(3), iconv(1), recode(1) AUTHOR
This manual page was written by Uwe Steinmann uwe@steinmann.cx. PX_SET_INPUTENCODING(3)
All times are GMT -4. The time now is 04:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy