Sponsored Content
Top Forums Shell Programming and Scripting Extracting a portion of data from a very large tab delimited text file Post 302412063 by alister on Sunday 11th of April 2010 11:55:58 AM
Old 04-11-2010
If Franklin52's solution does not work, perhaps it is because there are always five fields (four tabs delimiting possibly empty fields). In that case, you could try:
Code:
awk -F'\t' '{for (i=1;i<=NF;i++) if (!length($i)) next; print}' file

Regards,
Alister

Last edited by alister; 04-11-2010 at 03:44 PM.. Reason: Fix erroneous suggestions
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Check whether a given file is in ASCII format and data is tab-delimited

Hi All, Please help me out with a script which checks whether a given file say abc.txt is in ASCII format and data is tab-delimited. If the condition doesn't satisfy then it should generate error code "100" for file not in ASCII format and "105" if it is not in tab-delimited format. If the... (9 Replies)
Discussion started by: Mandab
9 Replies

2. Shell Programming and Scripting

Removing blanks in a text tab delimited file

Hi Experts I am very new to perl and need to make a script using perl. I would like to remove blanks in a text tab delimited file in in a specfic column range ( colum 21 to column 43) sample input and output shown below : Input: 117 102 650 652 654 656 117 93 95... (3 Replies)
Discussion started by: Faisal Riaz
3 Replies

3. Shell Programming and Scripting

Delete first column in tab-delimited text-file

I have a large text-file with tab-delimited genetic data that looks like: KSC112 KSC234 0 0 1 1 A G C T I simply wan to delete the first column, but since the file has 600 000 columns, it is not possible with awk (seems to be limited at 32k columns). Does anyone have an idea how to do this? (2 Replies)
Discussion started by: andmal
2 Replies

4. Shell Programming and Scripting

Extracting a portion of a data file with identifier

Hi, I do have a TAB delimted text file with the following format. 1 (- identifier of each group. this text is not present in the file only number) 1 3 4 65 56 WERTF 2 3 4 56 56 GHTYHU 3 3 5 64 23 VMFKLG 2 1 3 4 65 56 DGTEYDH 2 3 4 56 56 FJJJCKC 3 3 5 64 23 FNNNCHD 3 1 3 4 65 56 JDHJDH... (9 Replies)
Discussion started by: Lucky Ali
9 Replies

5. UNIX for Dummies Questions & Answers

How to convert text to columns in tab delimited text file

Hello Gurus, I have a text file containing nearly 12,000 tab delimited characters with 4000 rows. If the file size is small, excel can convert the text into coloumns. However, the file that I have is very big. Can some body help me in solving this problem? The input file example, ... (6 Replies)
Discussion started by: Unilearn
6 Replies

6. UNIX for Dummies Questions & Answers

Deleting columns from a tab delimited text file?

I have a tab limited text file with 10000+ columns. I want to delete columns 6 through 23, how do I go about doing that? Thanks! (1 Reply)
Discussion started by: evelibertine
1 Replies

7. UNIX for Dummies Questions & Answers

Add a new column to a tab delimited text file

I want to add a new column to a tab delimited text file. It will be the first column and it will just be 1's. How do I go about doing that? Thanks! (1 Reply)
Discussion started by: evelibertine
1 Replies

8. UNIX for Dummies Questions & Answers

How to convert a text file into tab delimited format?

I have a text file that made using text editor in Ubuntu. However the text file is not being recognized as space or tab delimited, the formatting seems to be messed up. How can I convert the text file into tab delimited format? (3 Replies)
Discussion started by: evelibertine
3 Replies

9. Shell Programming and Scripting

How to read data from tab delimited file after a specific position?

Hi Experts, I have a tab deliminated file as below myfile.txt Local Group Memberships *Administrators *Guests I need data in below format starting from 4th position. myfile1.txt Administrators Guests the above one is just an example and there could... (15 Replies)
Discussion started by: Litu1988
15 Replies

10. UNIX for Dummies Questions & Answers

Need to convert a pipe delimited text file to tab delimited

Hi, I have a rquirement in unix as below . I have a text file with me seperated by | symbol and i need to generate a excel file through unix commands/script so that each value will go to each column. ex: Input Text file: 1|A|apple 2|B|bottle excel file to be generated as output as... (9 Replies)
Discussion started by: raja kakitapall
9 Replies
INNWATCH.CTL(5) 						File Formats Manual						   INNWATCH.CTL(5)

NAME
innwatch.ctl - control Usenet supervision by innwatch DESCRIPTION
The file /etc/news/innwatch.ctl is used to determine what actions are taken during the periodic supervisions by innwatch. The file consists of a series of lines; blank lines and lines beginning with a number sign (``#'') are ignored. All other lines consist of seven fields, each preceded by a delimiting character: :label:state:condition:test:limit:command:reason The delimiter can be any one of several non-alphanumeric characters that does not appear elsewhere in the line; there is no way to quote it to include it in any of the fields. Any of ``!'', ``,'', ``:'', ``@'', ``;'', or ``?'' is a good choice. Each line can have a different delimiter; the first character on each line is the delimiter for that line. White space surrounding delimiters, except before the first, is ignored, and does not form part of the fields, white space within fields is permitted. All delimiters must be present. The first field is a label for the control line. It is used as an internal state indicator and in ctlinnd messages to control the server. If omitted, the line number is used. The second field specifies when this control line should be used. It consists of a list of labels, and special indicators, separated by whitespace. If the current state matches against any of the labels in this field, this line will be used as described below. The values that may be used are: - This line matches if the current state is the same as the label on this line, or if the current state is ``run,'' the initial state. This is also the default state if this field is empty. + This line matches if the current state is ``run.'' * This line always matches. label This line matches if the current state is the specified ``label.'' -label This line matches if the current state is not the specified ``label.'' The third field specifies a shell command that is invoked if this line matches. Do not use any shell filename expansion characters such as ``*'', ``?'', or ``['' (even quoted, they're not likely to work as intended). If the command succeeds, as indicated by its exit status, it is expected to have printed a single integer to standard output. This gives the value of this control line, to be used below. If the com- mand fails, the line is ignored. The command is executed with its current directory set to the news spool directory, /var/spool/news. The fourth field specifies the operator to use to test the value returned above. It should be one of the two letter numeric test operators defined in test(1) such as ``eq'', ``lt'' and the like. The leading dash (`'-'') should not be included. The fifth field specifies a constant with which to compare the value using the operator just defined. This is done by invoking the command test value -operator constant The line is said to ``succeed'' if it returns true. The sixth field specifies what should be done if the line succeeds, and in some cases if it fails. Any of the following words may be used: throttle Causes innwatch to throttle the server if this line succeeds. It also sets the state to the value of the line's label. If the line fails, and the state was previously equal to the label on this line (that is, this line had previously succeeded), then a go command will be sent to the server, and innwatch will return to the ``run'' state. The ``throttle'' is only performed if the current state is ``run'' or a state other than the label of this line, regardless of whether the command succeeds. pause Is identical to ``throttle'' except that the server is paused. shutdown Sends a ``shutdown'' command to the server. It is for emergency use only. flush Sends a ``flush'' command to the server. go Causes innwatch to send a ``go'' command to the server and to set the state to ``run.'' exit Causes innwatch to exit. skip The result of the control file is skipped for the current pass. The last field specifies the reason that is used in those ctlinnd commands that require one. More strictly, it is part of the reason -- innwatch appends some information to it. In order to enable other sites to recognize the state of the local innd server, this field should usually be set to one of several standard values. Use ``No space'' if the server is rejecting articles because of a lack of filesystem resources. Use ``loadav'' if the server is rejecting articles because of a lack of CPU resources. Once innwatch has taken some action as a consequence of its control line, it skips the rest of the control file for this pass. If the action was to restart the server (that is, issue a ``go'' command), then the next pass will commence almost immediately, so that innwatch can discover any other condition that may mean that the server should be suspended again. EXAMPLES
@@@df .|awk 'NR==2 {print $4}'@lt@10000@throttle@No space @@@df -i .|awk 'NR==2 {print $4}'@lt@1000@throttle@No space (inodes) The first line causes the server to be throttled if the free space drops below 10000 units (using whatever units df uses), and restarted again when free space increases above the threshold. The second line does the same for inodes. The next three lines act as a group and should appear in the following order. It is easier to explain them, however, if they are described from the last up. !load!load hiload!loadavg!lt!5!go! :hiload:+ load:loadavg:gt:8:throttle:loadav /load/+/loadavg/ge/6/pause/loadav The final line causes the server to be paused if innwatch is in the ``run'' state and the load average rises to, or above, six. The state is set to ``load'' when this happens. The previous line causes the server to be throttled when innwatch is in the ``run'' or ``load'' state, and the load average rises above eight. The state is set to ``hiload'' when this happens. Note that innwatch can switch the server from ``paused'' to ``throttled'' if the load average rises from below six to between six and seven, and then to above eight. The first line causes the server to be sent a ``go'' command if innwatch is in the ``load'' or ``hiload'' state, and the load average drops below five. Note that all three lines assume a mythical command loadavg that is assumed to print the current load average as an integer. In more prac- tical circumstances, a pipe of uptime into awk is more likely to be useful. BUGS
This file must be tailored for each individual site, the sample supplied is truly no more than a sample. The file should be ordered so that the more common problems are tested first. The ``run'' state is not actually identified by the label with that three letter name, and using it will not work as expected. Using an ``unusual'' character for the delimiter such as ``('', ``*'', ``&'', ```'', ``''', and the like, is likely to lead to obscure and hard to locate bugs. HISTORY
Written by <kre@munnari.oz.au> for InterNetNews. This is revision 1.5, dated 1996/09/06. SEE ALSO
innd(8), ctlinnd(8), news.daily(8). INNWATCH.CTL(5)
All times are GMT -4. The time now is 02:38 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy