How do I strip and add tabbed fields to a long text file? - Shell Programming and Scripting

saint65 · #1 (**permalink**) 02-20-2007

I was wondering if there was a way to modify a tab delimited text file, up to 185,000 lines long by adding a repeated field to a block of 20 data , then repeat the process until the file is processed.

The current file looks like:

Code:

/////////////////////////////////////////////////////////////////////////
//File Name:  
//Test Date and Time: 
//
//Test Information
//
Customer ID:
Company ID: 
Project ID: T6010
Test ID: OFAT7
Test Type:Test
Sheet : DP600-GI 1.2
Material: D2 -5mm
t: MO
Tester: 
Test Date: 06-12-20
//--------------------------------------------------------------------------
// DataLength
// TotalStroke GroupStroke
// PullSpeed(m/s)
//
//    Time(s)  Tin(°)     TDie(°)    Tout(°)  Pull_Force(N)
//--------------------------------------------------------------------------
20
1  1
0.124
      0.00      19.53      16.11      19.78     273.68
      0.05      16.85      16.11      19.53    4707.32
      0.10      19.04      16.36      19.29    8046.24
      0.15      19.04      16.60      19.29    8319.92
      0.20      19.53      16.36      19.53    9359.91
      0.25      18.31      16.60      19.53    9907.28
      0.30      19.04      16.11      19.53   11275.68
      0.35      18.80      16.36      19.29   12315.67
      0.40      19.04      16.60      19.29   13081.98
      0.45      19.04      16.85      19.29   13793.55
      0.50      19.04      16.36      19.29    9578.86
      0.55      18.80      16.60      19.29      54.74
      0.60      19.04      16.85      19.29     109.47
      0.65      19.29      16.60      19.29     164.21
      0.70      18.80      16.85      19.29     164.21
      0.75      18.80      16.85      19.78      54.74
      0.80      18.80      17.33      19.29     164.21
      0.85      18.80      17.33      19.04     109.47
      0.90      19.04      17.33      19.29     164.21
      0.95      19.04      17.58      19.29     109.47
2  2
0.124
      1.95      20.02      19.78      20.26     164.21
      2.00      19.78      20.26      20.26   13903.03
      2.05      19.78      19.78      20.51   14231.45
      2.10      19.78      19.78      20.26   13848.29
      2.15      19.78      20.26      20.02   13684.08
      2.20      20.02      20.02      20.26   13629.35
      2.25      20.02      20.26      20.02   13684.08
      2.30      19.78      20.26      20.26   13629.35
      2.35      19.78      20.51      20.26   13848.29
      2.40      20.02      20.51      20.26   13629.35
      2.45      19.78      20.75      20.02   13574.61
      2.50      19.78      20.51      20.02     383.15
      2.55      19.78      20.75      20.51     218.95
      2.60      19.78      21.00      20.26     109.47
      2.65      19.53      21.00      20.26     328.42
      2.70      20.02      21.00      20.26     218.95
      2.75      19.78      21.48      20.02     218.95
      2.80      19.78      21.73      20.51     218.95
      2.85      19.78      21.97      20.26     218.95
      2.90      19.78      21.73      20.26     273.68
3  3
0.124
      3.88      19.53      23.19      19.78     164.21
      3.92      19.29      23.19      19.78   14286.18
      3.98      19.29      23.44      20.02   14012.50
      4.03      19.29      22.95      20.02   13848.29
      4.08      19.29      23.19      19.78   13684.08
      4.13      19.53      23.44      20.02   13848.29
      4.17      19.53      23.19      19.78   13684.08
      4.22      19.78      23.68      19.78   13684.08
      4.28      19.78      23.68      19.78   13684.08
      4.33      19.53      23.68      20.02   13684.08
      4.38      19.53      23.93      20.02   13519.87
      4.42      19.29      23.93      20.02     109.47
      4.47      19.29      23.93      19.78     218.95
      4.53      19.53      23.93      19.78     164.21
      4.58      19.53      24.17      19.53     164.21
      4.63      19.53      24.17      19.53     164.21
      4.67      19.53      24.66      19.78     164.21
      4.72      19.53      24.41      20.02     218.95
      4.78      19.53      24.17      20.26     164.21
      4.83      19.29      24.90      19.78     164.21
4  4
0.124
      5.80      18.55      24.90      19.04      54.74

I wish to reorganize the file to remove the top 27 lines and format like the following, remember that there are up to 9000 sample times 20 points of data:

Code:

//1  1
//0.124
1  1  0.124      0.00      19.53      16.11      19.78     273.68
1  1  0.124      0.05      16.85      16.11      19.53    4707.32
1  1  0.124      0.10      19.04      16.36      19.29    8046.24
1  1  0.124      0.15      19.04      16.60      19.29    8319.92
1  1  0.124      0.20      19.53      16.36      19.53    9359.91
1  1  0.124      0.25      18.31      16.60      19.53    9907.28
1  1  0.124      0.30      19.04      16.11      19.53   11275.68
1  1  0.124      0.35      18.80      16.36      19.29   12315.67
1  1  0.124      0.40      19.04      16.60      19.29   13081.98
1  1  0.124      0.45      19.04      16.85      19.29   13793.55
1  1  0.124      0.50      19.04      16.36      19.29    9578.86
1  1  0.124      0.55      18.80      16.60      19.29      54.74
1  1  0.124      0.60      19.04      16.85      19.29     109.47
1  1  0.124      0.65      19.29      16.60      19.29     164.21
1  1  0.124      0.70      18.80      16.85      19.29     164.21
1  1  0.124      0.75      18.80      16.85      19.78      54.74
1  1  0.124      0.80      18.80      17.33      19.29     164.21
1  1  0.124      0.85      18.80      17.33      19.04     109.47
1  1  0.124      0.90      19.04      17.33      19.29     164.21
1  1  0.124      0.95      19.04      17.58      19.29     109.47
//2  2
//0.124
2  2  0.124      1.95      20.02      19.78      20.26     164.21
2  2  0.124      2.00      19.78      20.26      20.26   13903.03
2  2  0.124      2.05      19.78      19.78      20.51   14231.45
2  2  0.124      2.10      19.78      19.78      20.26   13848.29
2  2  0.124      2.15      19.78      20.26      20.02   13684.08
2  2  0.124      2.20      20.02      20.02      20.26   13629.35
2  2  0.124      2.25      20.02      20.26      20.02   13684.08
2  2  0.124      2.30      19.78      20.26      20.26   13629.35
2  2  0.124      2.35      19.78      20.51      20.26   13848.29
2  2  0.124      2.40      20.02      20.51      20.26   13629.35
2  2  0.124      2.45      19.78      20.75      20.02   13574.61
2  2  0.124      2.50      19.78      20.51      20.02     383.15
2  2  0.124      2.55      19.78      20.75      20.51     218.95
2  2  0.124      2.60      19.78      21.00      20.26     109.47
2  2  0.124      2.65      19.53      21.00      20.26     328.42
2  2  0.124      2.70      20.02      21.00      20.26     218.95
2  2  0.124      2.75      19.78      21.48      20.02     218.95
2  2  0.124      2.80      19.78      21.73      20.51     218.95
2  2  0.124      2.85      19.78      21.97      20.26     218.95
2  2  0.124      2.90      19.78      21.73      20.26     273.68
//3  3
//0.124
3  3  0.124      3.88      19.53      23.19      19.78     164.21
3  3  0.124      3.92      19.29      23.19      19.78   14286.18
3  3  0.124      3.98      19.29      23.44      20.02   14012.50
3  3  0.124      4.03      19.29      22.95      20.02   13848.29
3  3  0.124      4.08      19.29      23.19      19.78   13684.08
3  3  0.124      4.13      19.53      23.44      20.02   13848.29
3  3  0.124      4.17      19.53      23.19      19.78   13684.08
3  3  0.124      4.22      19.78      23.68      19.78   13684.08
3  3  0.124      4.28      19.78      23.68      19.78   13684.08
3  3  0.124      4.33      19.53      23.68      20.02   13684.08
3  3  0.124      4.38      19.53      23.93      20.02   13519.87
3  3  0.124      4.42      19.29      23.93      20.02     109.47
3  3  0.124      4.47      19.29      23.93      19.78     218.95
3  3  0.124      4.53      19.53      23.93      19.78     164.21
3  3  0.124      4.58      19.53      24.17      19.53     164.21
3  3  0.124      4.63      19.53      24.17      19.53     164.21
3  3  0.124      4.67      19.53      24.66      19.78     164.21
3  3  0.124      4.72      19.53      24.41      20.02     218.95
3  3  0.124      4.78      19.53      24.17      20.26     164.21
3  3  0.124      4.83      19.29      24.90      19.78     164.21
//4  4
//0.124
4  4  0.124      5.80      18.55      24.90      19.04      54.74

Any suggestions of how I could approach this would be greatly appreciated. I just do not know where to start.

anbu23 · #2 (**permalink**) 02-20-2007

Code:

awk '
NR > 24 { 
if ( NF == 2 ) 
{ 
	a=$1;b=$2; print "//"$0 ; 
	getline c; print "//"c ; 
	getline;
}
print a "\t" b "\t" c "\t" $0 } ' file

saint65 · #3 (**permalink**) 02-20-2007

That works fantastically. Thanks so much, I tried

Code:

awk '
NR > 24 { 
if ( NF == 2 ) 
{ 
	a=$1;b=$2; print "//"$0 ; 
	getline c; print "//"c ; 
	getline;
}
print a "\t" b "\t" c "\t" $0 } ' file

which dumped to screen, I added

Code:

awk '
NR > 24 { 
if ( NF == 2 ) 
{ 
	a=$1;b=$2; print "//"$0 ; 
	getline c; print "//"c ; 
	getline;
}
print a "\t" b "\t" c "\t" $0 } ' file > xyz

to put it into a file.

If i was to add the contents of a second data file that has to be added in sequence, how would i change the numbering?

The first digit (field) is the total count 1-9000 (file 1) the second digit (second field) is group count (just used but not totalled), I need to make 1-9000 (first field) of second data file go from 9001-18000 and so on.

I will be adding contents of file, file1, file2, ... file_n ---> xyz

anbu23 · #4 (**permalink**) 02-20-2007

Code:

awk '
FNR > 24 { 
if ( NF == 2 ) 
{ 
	a=$1;b=$2; print "//"$0 ; 
	getline c; print "//"c ; 
	getline;
}
print a "\t" b "\t" c "\t" $0 } ' file1 file2 file3  > xyz

If the format of the files are same, then you can use the above code

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
combining fields in two text fields	shocker	Shell Programming and Scripting	3	01-16-2008 07:27 AM
how to strip rows from a text file?	ihot	Shell Programming and Scripting	5	10-16-2007 06:21 PM
Extracting information from text fields.	spindoctor	UNIX for Dummies Questions & Answers	24	06-09-2007 10:17 AM
Need a Help with sort a text file with some fields	alexcol	Shell Programming and Scripting	3	02-18-2007 11:13 PM
Find/Change long text	aungomarin	Shell Programming and Scripting	1	07-10-2006 02:45 AM

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode