Sponsored Content
Top Forums Shell Programming and Scripting formatting data file with awk or sed Post 302407710 by lego on Friday 26th of March 2010 06:51:30 AM
Old 03-26-2010
Thanks for your replies, but I couldn't get it to work with my file..
I attach here the file I need to work with (data.txt), and I'll try to explain what I exactly want.
I have a file which has this structure:

Quote:
header
more header
more header..
x1 x2 x3 x4 x5 x6
x7 x8 ...
...
... x59 x60
y1 y2 y3 y4 y5 y6
y7 y8 ...
...
... y99 y100
y1 y2 y3 y4 y5 y6
y7 y8 ...
...
... y99 y100
p1 p2 p3 p4 p5 p6
p7 p8 ...
...
... p59 p60
d1 d2 d3 d4 d5 d6
d7 d8 ...
...
...d59 d60
e1 e2 e3 e4 e5 e6
e7 e8 ...
...
... e59 e60
f1 f2 f3 f4 f5 f6
f7 f8 ...
...
... f59 f60
p61 p62 p63 p64 p65 p66
p67 p68 ...
...
...p119 p120
d61 d62 d63 d64 d65 d66
d67 d68 ...
...
...d119 d120
e61 e62 e63 e64 e65 e66
e67 e68 ...
...
... e119 e120
f61 f62 f63 f64 f65 f66
f67 f68 ...
...
... f119 f120
p121 p122 p123 ...
...
...
...
...f5999 f6000
What I need is this output file:

Quote:
x1 y1 p1 e1 d1 f1
x1 y2 p2 e2 d2 f2
x1 y3 p3 e3 d3 f3
...
x1 y100 p100 e100 d100 f100

x2 y1 p101 e101 d101 f101
x2 y2 p102 e102 d102 f102
x2 y3 p103 e103 d103 f103
...
x2 y100 p200 e200 d200 f200

x3 y1 p201 e201 d201 f201
x3 y2 p202 e202 d202 f202
...
...
x60 y1 p5901 e5901 d5901 f5901
...
x60 y100 p6000 e6000 d6000 f6000
In the attached file, the header spans 14 columns. From lines 15 to 24 (inclusive) there are the X values. From 25 to 41 (and also 42 to 58) there are the Y values.
From line 61 to 70, there are the first 60 P values, from 71 to 80 the first E values, ...
From line 101 to 110, there are the values of P61, P62, ....

In total, there are 60 different values for X, 100 different values for Y, and 6000 different values for P,E,D, and F.

I hope I have explained my problem clearly.
Please help me!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Clense Junk Data File - Using Shell or awk or sed

Hello Shell Gurus i need help in solving this puzzle. We have a junk data file that needs to be fed into the database. Need to clense the data file thru shell script. I am not a expert and so need help with Here is what i need to do on the input file -Step -1 Replace all pipes ‘|' within... (1 Reply)
Discussion started by: rimss
1 Replies

2. Shell Programming and Scripting

Big data file - sed/grep/awk?

Morning guys. Another day another question. :rolleyes: I am knocking up a script to pull some data from a file. The problem is the file is very big (up to 1 gig in size), so this solution: for results in `grep "^\ ... works, but takes ages (we're talking minutes) to run. The data is held... (8 Replies)
Discussion started by: dlam
8 Replies

3. Shell Programming and Scripting

Awk formatting of a data file - nested for loops?

Hello - is there any way in awk I can do... 4861 x(1) y(1) z(1) 4959 x(1) y(1) z(1) 5007 x(1) y(1) z(1) 4861 x(2) y(2) z(2) 4959 x(2) y(2) z(2) 5007 x(2) y(2) z(2) 4861 x(3) y(3) z(3) 4959 x(3) y(3) z(3) 5007 x(3) y(3) z(3) to become... 4861 x(1) y(1) z(1) 4861 x(2) y(2) z(2)... (3 Replies)
Discussion started by: catwoman
3 Replies

4. Shell Programming and Scripting

sed or awk to extract data from Xml file

Hi, I want to get data from Xml file by using sed or awk command. I want to get the following result : mon titre 1;Createur1;Dossier1 mon titre 1;Createur1;Dossier1 and save it in cvs file (fichier.cvs). FROM this Xml file (test.xml): <playlist version="1"> <trackList> <track>... (1 Reply)
Discussion started by: yeclota
1 Replies

5. Shell Programming and Scripting

Formatting help needed awk or sed maybe

I am executing the following command: sort file1.txt | uniq -c | sort -n > file2.txt The problem is that in file 2, I get leading spaces, Like so: 1 N/A|A8MW11 8 N/A|ufwo1 9 N/A|a8mw11 10 900003|smoketest297688 10 N/A|a9dg4 10 danny|danni 12... (5 Replies)
Discussion started by: ddurden7
5 Replies

6. Shell Programming and Scripting

how to get data from hex file using SED or AWK based on pattern sign

I have a binary (hex) file I need to parse to get some data which are encoded this way: .* b4 . . . 01 12 .* af .* 83 L1 x1 x2 xL 84 L2 y1 y2 yL By another words there is a stream of hexadecimal bytes (in my example separated by space for better readability). I need to get value stored in... (3 Replies)
Discussion started by: sameucho
3 Replies

7. Shell Programming and Scripting

AWK/Shell script for formatting data in a file

Hi All, Need an urgent help to convert a unix file in to a particular format: **source file:** 1111111 2d2f2h2 3dfgsd3 ........... 1111111 <-- repeats in every nth line. remaining all lines will be different 123ss41 432ff45 ........... 1111111 <-- repetition qwe1234 123weq3... (1 Reply)
Discussion started by: rajivnairfis
1 Replies

8. Shell Programming and Scripting

awk - sed / reading from a data file and doing algebraic operations

Hi everyone, I am trying to write a bash script which reads a data file and does some algebraic operations. here is the structure of data.xml file that I have; 1 <data> 2 . 3 . 4 . 5 </data> 6 <data> 7 . 8 . 9 . 10</data> etc. Each data block contains same number of lines (say... (4 Replies)
Discussion started by: hayreter
4 Replies

9. Shell Programming and Scripting

Data formatting using awk

Need assistance on the data extraction using awk Below is the format and would like to extract the data in another format ------------------------------------------------------------------------------------------------- Minimum Temperature (deg F ) DAY 1 2 3 4 5 6 7 8 9 10 11... (4 Replies)
Discussion started by: ajayram_arya
4 Replies

10. Shell Programming and Scripting

Problem in formatting output in sed / awk

I have a file like this : ! 1 ! 542255 ! 50,140.00 ! ! 2 ! 551717 ! 5,805.00 ! ! 3 ! 551763 ! 8,130.00 ! ! 4 ! 551779 ! 750.00 ! ! 5 ! 551810 ! 56,580.00 ! ! 6 ! 551816 ! 1,350.00 ! ! 7 ! 551876 ! 360.00 ! ! 8 ! 551898 ! ... (10 Replies)
Discussion started by: adam1969in
10 Replies
HEADERS_SENT(3) 							 1							   HEADERS_SENT(3)

headers_sent - Checks if or where headers have been sent

SYNOPSIS
bool headers_sent ([string &$file], [int &$line]) DESCRIPTION
Checks if or where headers have been sent. You can't add any more header lines using the header(3) function once the header block has already been sent. Using this function you can at least prevent getting HTTP header related error messages. Another option is to use Output Buffering. PARAMETERS
o $file - If the optional $file and $line parameters are set, headers_sent(3) will put the PHP source file name and line number where out- put started in the $file and $line variables. o $line - The line number where the output started. RETURN VALUES
headers_sent(3) will return FALSE if no HTTP headers have already been sent or TRUE otherwise. EXAMPLES
Example #1 Examples using headers_sent(3) <?php // If no headers are sent, send one if (!headers_sent()) { header('Location: http://www.example.com/'); exit; } // An example using the optional file and line parameters, as of PHP 4.3.0 // Note that $filename and $linenum are passed in for later use. // Do not assign them values beforehand. if (!headers_sent($filename, $linenum)) { header('Location: http://www.example.com/'); exit; // You would most likely trigger an error here. } else { echo "Headers already sent in $filename on line $linenum " . "Cannot redirect, for now please click this <a " . "href="http://www.example.com">link</a> instead "; exit; } ?> NOTES
Note Headers will only be accessible and output when a SAPI that supports them is in use. SEE ALSO
ob_start(3), trigger_error(3), headers_list(3), header(3) for a more detailed discussion of the matters involved. . PHP Documentation Group HEADERS_SENT(3)
All times are GMT -4. The time now is 09:52 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy