Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How do I strip and add tabbed fields to a long text file?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How do I strip and add tabbed fields to a long text file?
# 1  
Old 02-20-2007
How do I strip and add tabbed fields to a long text file?
I was wondering if there was a way to modify a tab delimited text file, up to 185,000 lines long by adding a repeated field to a block of 20 data , then repeat the process until the file is processed.

The current file looks like:
Code:
/////////////////////////////////////////////////////////////////////////
//File Name:  
//Test Date and Time: 
//
//Test Information
//
Customer ID:
Company ID: 
Project ID: T6010
Test ID: OFAT7
Test Type:Test
Sheet : DP600-GI 1.2
Material: D2 -5mm
t: MO
Tester: 
Test Date: 06-12-20
//--------------------------------------------------------------------------
// DataLength
// TotalStroke GroupStroke
// PullSpeed(m/s)
//
//    Time(s)  Tin(°)     TDie(°)    Tout(°)  Pull_Force(N)
//--------------------------------------------------------------------------
20
1  1
0.124
      0.00      19.53      16.11      19.78     273.68
      0.05      16.85      16.11      19.53    4707.32
      0.10      19.04      16.36      19.29    8046.24
      0.15      19.04      16.60      19.29    8319.92
      0.20      19.53      16.36      19.53    9359.91
      0.25      18.31      16.60      19.53    9907.28
      0.30      19.04      16.11      19.53   11275.68
      0.35      18.80      16.36      19.29   12315.67
      0.40      19.04      16.60      19.29   13081.98
      0.45      19.04      16.85      19.29   13793.55
      0.50      19.04      16.36      19.29    9578.86
      0.55      18.80      16.60      19.29      54.74
      0.60      19.04      16.85      19.29     109.47
      0.65      19.29      16.60      19.29     164.21
      0.70      18.80      16.85      19.29     164.21
      0.75      18.80      16.85      19.78      54.74
      0.80      18.80      17.33      19.29     164.21
      0.85      18.80      17.33      19.04     109.47
      0.90      19.04      17.33      19.29     164.21
      0.95      19.04      17.58      19.29     109.47
2  2
0.124
      1.95      20.02      19.78      20.26     164.21
      2.00      19.78      20.26      20.26   13903.03
      2.05      19.78      19.78      20.51   14231.45
      2.10      19.78      19.78      20.26   13848.29
      2.15      19.78      20.26      20.02   13684.08
      2.20      20.02      20.02      20.26   13629.35
      2.25      20.02      20.26      20.02   13684.08
      2.30      19.78      20.26      20.26   13629.35
      2.35      19.78      20.51      20.26   13848.29
      2.40      20.02      20.51      20.26   13629.35
      2.45      19.78      20.75      20.02   13574.61
      2.50      19.78      20.51      20.02     383.15
      2.55      19.78      20.75      20.51     218.95
      2.60      19.78      21.00      20.26     109.47
      2.65      19.53      21.00      20.26     328.42
      2.70      20.02      21.00      20.26     218.95
      2.75      19.78      21.48      20.02     218.95
      2.80      19.78      21.73      20.51     218.95
      2.85      19.78      21.97      20.26     218.95
      2.90      19.78      21.73      20.26     273.68
3  3
0.124
      3.88      19.53      23.19      19.78     164.21
      3.92      19.29      23.19      19.78   14286.18
      3.98      19.29      23.44      20.02   14012.50
      4.03      19.29      22.95      20.02   13848.29
      4.08      19.29      23.19      19.78   13684.08
      4.13      19.53      23.44      20.02   13848.29
      4.17      19.53      23.19      19.78   13684.08
      4.22      19.78      23.68      19.78   13684.08
      4.28      19.78      23.68      19.78   13684.08
      4.33      19.53      23.68      20.02   13684.08
      4.38      19.53      23.93      20.02   13519.87
      4.42      19.29      23.93      20.02     109.47
      4.47      19.29      23.93      19.78     218.95
      4.53      19.53      23.93      19.78     164.21
      4.58      19.53      24.17      19.53     164.21
      4.63      19.53      24.17      19.53     164.21
      4.67      19.53      24.66      19.78     164.21
      4.72      19.53      24.41      20.02     218.95
      4.78      19.53      24.17      20.26     164.21
      4.83      19.29      24.90      19.78     164.21
4  4
0.124
      5.80      18.55      24.90      19.04      54.74

I wish to reorganize the file to remove the top 27 lines and format like the following, remember that there are up to 9000 sample times 20 points of data:

Code:
//1  1
//0.124
1  1  0.124      0.00      19.53      16.11      19.78     273.68
1  1  0.124      0.05      16.85      16.11      19.53    4707.32
1  1  0.124      0.10      19.04      16.36      19.29    8046.24
1  1  0.124      0.15      19.04      16.60      19.29    8319.92
1  1  0.124      0.20      19.53      16.36      19.53    9359.91
1  1  0.124      0.25      18.31      16.60      19.53    9907.28
1  1  0.124      0.30      19.04      16.11      19.53   11275.68
1  1  0.124      0.35      18.80      16.36      19.29   12315.67
1  1  0.124      0.40      19.04      16.60      19.29   13081.98
1  1  0.124      0.45      19.04      16.85      19.29   13793.55
1  1  0.124      0.50      19.04      16.36      19.29    9578.86
1  1  0.124      0.55      18.80      16.60      19.29      54.74
1  1  0.124      0.60      19.04      16.85      19.29     109.47
1  1  0.124      0.65      19.29      16.60      19.29     164.21
1  1  0.124      0.70      18.80      16.85      19.29     164.21
1  1  0.124      0.75      18.80      16.85      19.78      54.74
1  1  0.124      0.80      18.80      17.33      19.29     164.21
1  1  0.124      0.85      18.80      17.33      19.04     109.47
1  1  0.124      0.90      19.04      17.33      19.29     164.21
1  1  0.124      0.95      19.04      17.58      19.29     109.47
//2  2
//0.124
2  2  0.124      1.95      20.02      19.78      20.26     164.21
2  2  0.124      2.00      19.78      20.26      20.26   13903.03
2  2  0.124      2.05      19.78      19.78      20.51   14231.45
2  2  0.124      2.10      19.78      19.78      20.26   13848.29
2  2  0.124      2.15      19.78      20.26      20.02   13684.08
2  2  0.124      2.20      20.02      20.02      20.26   13629.35
2  2  0.124      2.25      20.02      20.26      20.02   13684.08
2  2  0.124      2.30      19.78      20.26      20.26   13629.35
2  2  0.124      2.35      19.78      20.51      20.26   13848.29
2  2  0.124      2.40      20.02      20.51      20.26   13629.35
2  2  0.124      2.45      19.78      20.75      20.02   13574.61
2  2  0.124      2.50      19.78      20.51      20.02     383.15
2  2  0.124      2.55      19.78      20.75      20.51     218.95
2  2  0.124      2.60      19.78      21.00      20.26     109.47
2  2  0.124      2.65      19.53      21.00      20.26     328.42
2  2  0.124      2.70      20.02      21.00      20.26     218.95
2  2  0.124      2.75      19.78      21.48      20.02     218.95
2  2  0.124      2.80      19.78      21.73      20.51     218.95
2  2  0.124      2.85      19.78      21.97      20.26     218.95
2  2  0.124      2.90      19.78      21.73      20.26     273.68
//3  3
//0.124
3  3  0.124      3.88      19.53      23.19      19.78     164.21
3  3  0.124      3.92      19.29      23.19      19.78   14286.18
3  3  0.124      3.98      19.29      23.44      20.02   14012.50
3  3  0.124      4.03      19.29      22.95      20.02   13848.29
3  3  0.124      4.08      19.29      23.19      19.78   13684.08
3  3  0.124      4.13      19.53      23.44      20.02   13848.29
3  3  0.124      4.17      19.53      23.19      19.78   13684.08
3  3  0.124      4.22      19.78      23.68      19.78   13684.08
3  3  0.124      4.28      19.78      23.68      19.78   13684.08
3  3  0.124      4.33      19.53      23.68      20.02   13684.08
3  3  0.124      4.38      19.53      23.93      20.02   13519.87
3  3  0.124      4.42      19.29      23.93      20.02     109.47
3  3  0.124      4.47      19.29      23.93      19.78     218.95
3  3  0.124      4.53      19.53      23.93      19.78     164.21
3  3  0.124      4.58      19.53      24.17      19.53     164.21
3  3  0.124      4.63      19.53      24.17      19.53     164.21
3  3  0.124      4.67      19.53      24.66      19.78     164.21
3  3  0.124      4.72      19.53      24.41      20.02     218.95
3  3  0.124      4.78      19.53      24.17      20.26     164.21
3  3  0.124      4.83      19.29      24.90      19.78     164.21
//4  4
//0.124
4  4  0.124      5.80      18.55      24.90      19.04      54.74

Any suggestions of how I could approach this would be greatly appreciated. I just do not know where to start.
# 2  
Old 02-20-2007
Code:
awk '
NR > 24 { 
if ( NF == 2 ) 
{ 
	a=$1;b=$2; print "//"$0 ; 
	getline c; print "//"c ; 
	getline;
}
print a "\t" b "\t" c "\t" $0 } ' file

# 3  
Old 02-20-2007
That works fantastically. Thanks so much, I tried

Code:
awk '
NR > 24 { 
if ( NF == 2 ) 
{ 
	a=$1;b=$2; print "//"$0 ; 
	getline c; print "//"c ; 
	getline;
}
print a "\t" b "\t" c "\t" $0 } ' file

which dumped to screen, I added
Code:
awk '
NR > 24 { 
if ( NF == 2 ) 
{ 
	a=$1;b=$2; print "//"$0 ; 
	getline c; print "//"c ; 
	getline;
}
print a "\t" b "\t" c "\t" $0 } ' file > xyz

to put it into a file.

If i was to add the contents of a second data file that has to be added in sequence, how would i change the numbering?

The first digit (field) is the total count 1-9000 (file 1) the second digit (second field) is group count (just used but not totalled), I need to make 1-9000 (first field) of second data file go from 9001-18000 and so on.

I will be adding contents of file, file1, file2, ... file_n ---> xyz
# 4  
Old 02-21-2007
Code:
awk '
FNR > 24 { 
if ( NF == 2 ) 
{ 
	a=$1;b=$2; print "//"$0 ; 
	getline c; print "//"c ; 
	getline;
}
print a "\t" b "\t" c "\t" $0 } ' file1 file2 file3  > xyz

If the format of the files are same, then you can use the above code
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Tabbed multiple csv files into one single excel file with using shell script not perl

Hi Experts, I am querying backup status results for multiple databases and getting each and every database result in one csv file. so i need to combine all csv files in one excel file with separate tabs. I am not familiar with perl script so i am using shell script. Could anyone please... (4 Replies)
Discussion started by: ramakrk2
4 Replies

2. UNIX for Dummies Questions & Answers

Editing long text file

Good morning all, I have a machine running IRIX and I need to edit a text file on the terminal that is literally thousands of lines. Does anyone know the most efficient way to edit portions of files like these? Obviously simply using the vi command isn't going to work since I get a too many lines... (1 Reply)
Discussion started by: James C
1 Replies

3. Shell Programming and Scripting

Strip the first record off a file and isolate fields off of it

I have a 2 part question on how to this in unix scripting using kshell or c shell. I have a file described below: 1st record has 2 fields on it every other record has 22 fields on it. Example ABC, email address Detail 1 Detail 2 Detail 3 . . . 1st question is... (4 Replies)
Discussion started by: jclanc8
4 Replies

4. Shell Programming and Scripting

Parse file for fields and specific text

I have a file of ~500,000 entries in the following: file.txt chr1 11868 12227 ENSG00000223972.5 . + HAVANA exon . gene_id "ENSG00000223972.5"; transcript_id "ENST00000456328.2"; gene_type "transcribed_unprocessed_pseudogene"; gene_status "KNOWN"; gene_name "DDX11L1"; transcript_type... (17 Replies)
Discussion started by: cmccabe
17 Replies

5. Shell Programming and Scripting

Strip some text and format with new delimited

Sample input 19:08:12.172; Cat1 74598; Cat2 1366; Cat3 227; Cat4 389; Cat5 572; Cat6 2228; Cat7 1039; Cat8 25; 19:08:22.173; Cat1 75589; Cat2 1388; Cat3 233; Cat4 393; Cat5 582; Cat6 2253; Cat7 1055; Cat8 25; 19:08:32.173; Cat1 76518; Cat2 1404; Cat3 238; Cat4 397; Cat5 592; Cat6 2270; Cat7... (5 Replies)
Discussion started by: before4
5 Replies

6. UNIX and Linux Applications

tabbed text editor without big libraries

I am looking for a tabbed text editor without a big library like gnome, kde, and gtk, I know about gedit, kate with extensions, geany, and bluefish. I would prefer it to be like gedit and be really light weight. So if anyone knows of a text editor that doesn't require those big libraries please let... (3 Replies)
Discussion started by: cokedude
3 Replies

7. Shell Programming and Scripting

Getting required fields from a text file in UNIX

My data is something like as shown below. Out of this i want the details of alarms (ex: 1947147711,1947147081......) and the fields( ex :sw=tacmwafabb9:shelf=1:slot=5-2:pport=2) Once i have these details separated, i want the count of these excluding the duplicates. What is the best possible way... (7 Replies)
Discussion started by: rdhanek
7 Replies

8. UNIX for Dummies Questions & Answers

extract fields from text file using delimiter!!

Hi All, I am new to unix scripting, please help me in solving this assignment.. I have a scenario, as follows: 1. i have a text file(read1.txt) with the following data sairam,123 kamal,122 etc.. 2. I have to write a unix... (6 Replies)
Discussion started by: G.K.K
6 Replies

9. Shell Programming and Scripting

how to strip rows from a text file?

Can an expert kindly write an efficient Linux ksh script that will strip rows with no numbers from a text file? Supposing there are three rows that text file called text.txt : "field1","field2","field3",11,22,33,44 "field1","field2","field3",1,2,3,4 "field1","field2","field3",,,, The... (5 Replies)
Discussion started by: ihot
5 Replies

10. Shell Programming and Scripting

Need a Help with sort a text file with some fields

Ive got a file called listacdrs with this structure: 01/09/2006 12:13 p.m. 1.046.528 CF0155.DAT 01/09/2006 12:13 p.m. 1.046.528 CF0156.DAT 01/09/2006 12:13 p.m. 1.046.528 CF0157.DAT 01/09/2006 12:13 p.m. 1.046.528 CF0158.DAT 01/09/2006 12:14 p.m. ... (3 Replies)
Discussion started by: alexcol
3 Replies
Login or Register to Ask a Question