Sponsored Content
Top Forums Shell Programming and Scripting Removing repeating lines from a data frame (AWK) Post 302539921 by gd9629 on Tuesday 19th of July 2011 06:04:05 AM
Old 07-19-2011
sorry I edited my original post to make it clearer. Yes the .csv has all the files and headers aligned, I just need to remove all the text (except the very first line) in the .csv file so that I can process the data.

grep doesn't seem to work? as I'm editing the file using GAWK? I'm pretty new to programming so I'm not sure how to find my way around this apparent conflict lol.

bartus, that code just added the first two column headers together and put them after the rest of the other headers.

Any ideas?

Cheers!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Omit repeating lines

Can someone help me with the following 2 objectives? 1) The following command is just an example. It gets a list of all print jobs. From there I am trying to extract the printer name. It works with the following command: lpstat -W "completed" -o | awk -F- '{ print $1}' Problem is, I want... (6 Replies)
Discussion started by: TheCrunge
6 Replies

2. Shell Programming and Scripting

frame multiple lines into one

Hi, i have a file with contents like below ( any number of entries can be there) 111 222 333 444 555 i need to make another file with single line like below: 111,222,333,444,555 (without ending , ) TIA Prvn (8 Replies)
Discussion started by: prvnrk
8 Replies

3. UNIX for Advanced & Expert Users

removing frame charecters

Hi I have a requirement as follows. My Input file is as follows. COL1,COL2,COL3,COL4,COL5 987,2,3~7~5,400~468~598,0005~4687~5980 1111,2,2~7,400~468,0005~897 Expected OUTPUT ============ COL1,COL2,COL3,COL4,COL5 987,2,3,400,0005 987,2,7,468,4687 987,2,5,598,5980 1111,2,2,400,0005... (6 Replies)
Discussion started by: tkbharani
6 Replies

4. Shell Programming and Scripting

Merging non-repeating columns of lines

Hello, I have file to work with. It has 5 columns. The first three, altogether, constitutes the position. The 4th column contains some values for downstream analysis and the fifth column contains some values that I want to add to 4th column (only if they happen to be in the same position). My... (5 Replies)
Discussion started by: menenuh
5 Replies

5. UNIX for Dummies Questions & Answers

Extract repeating data from file

I want to extract the last rows of a data file, similar to that one below: C1 xxx C2 rrr C3 ttt .... Cn-1 hhh Cn bbb C1 yyy C2 sss C3 uuu ... Cn-1 iii Cn ccc ... I just want to extract the final rows between C1 and Cn at each data file. n is not a constant,... (2 Replies)
Discussion started by: natasha
2 Replies

6. Shell Programming and Scripting

awk removing data before or after a pattern

I have the following data: 01:00:00 29 10 20 41 01:20:00 18 6 34 42 01:40:00 28 5 24 43 02:00:01 11 7 8 74 02:20:01 19 15 12 54 02:40:01 1 4 0 95 03:00:01 1... (6 Replies)
Discussion started by: BeefStu
6 Replies

7. UNIX for Dummies Questions & Answers

Remove groups of repeating lines

I know uniq exists, but am not sure how to remove repeating lines when they are groups of two different lines repeating themselves, without using sort. I need them to be sorted in the original order, just to remove repeats. cd /media/AUDIO/WAVE/9780743518673/mp3 ~/Desktop/mp3-to-m4b... (1 Reply)
Discussion started by: glev2005
1 Replies

8. Shell Programming and Scripting

perform actions at specific locations in data frame

Hi everyone, I got a data frame like the one below and and would like to do the following: Ignore the first 3 rows and check in all following rows the second position. If the value is >500, subtract 100. Example DF: ABC 22 DE 12 BCD 223 GH 12 EFG 2104 DH ... (4 Replies)
Discussion started by: TuAd
4 Replies

9. Shell Programming and Scripting

awk : collecting all data between two time frame

Hi Experts , I need your help to collect the complete data between two time frame from the log files, when I try awk it's collecting the data only which is printed with time stamp for example, awk works well from "16:00 to 17:30" but its not collecting <line*> "from 17:30 to 18:00" ... (8 Replies)
Discussion started by: zenkarthi
8 Replies

10. Shell Programming and Scripting

Removing certain lines from results - awk

im using the code below to monitor a file: gawk '{ a += gsub("(^| )accepted( |$)", "&") a += gsub("(^| )open database( |$)", "&") } END { for (i in a) printf("%s=%s\n", i, a) }' /var/log/syslog the code is searching the syslog file for the string "accepted" and "open... (2 Replies)
Discussion started by: SkySmart
2 Replies
HEADACHE(1)							  [FIXME: manual]						       HEADACHE(1)

NAME
headache - A program to manage the license of your source file. SYNOPSIS
headache [-h file] [-c file] [-r] [-help | --help] file... DESCRIPTION
This manual page documents briefly the headache command. This manual page was written for the Debian GNU/Linux distribution because the original program does not have a manual page. It is a common usage to put at the beginning of source code files a short header giving, for instance, some copyright information. headache is a simple and lightweight tool for managing easily these headers. Among its functionalities, one may mention: o Headers must generally be generated as comments in source code files. headache deals with different files types and generates for each of them headers in an appropriate format. o Headers automatically detects existing headers and removes them. Thus, you can use it to update headers in a set of files. COMMAND LINE OPTIONS
file Name of one file to process -h file Create a header with text coming from file -c file Read the given configuration file -help, --help Display the option list of headache USAGE
Let us illustrate the use of this tool with a small example. Assume you have a small project mixing C and Caml code consisting in three files 'foo.c', 'bar.ml' and 'bar.mli'', and you want to equip them with some header. First of all, write a header file, i.e. a plain text file including the information headers must mention. An example of such a file is given in figure 1. In the following, we assume this file is named 'myheader' and is in the same directory as source files. Then, in order to generate headers, just run the command : headache -h myheader foo.c bar.ml bar.mli Each file is equipped with an header including the text given in the header file 'myheader', surrounded by some extra characters depending on its format making it a comment (e.g. '(*' and '*)' in '.ml' files). If you update the header file 'myheader', you simply need to re-run the above command to update headers in source code files. Similarly, running : headache -r foo.c bar.ml bar.mli removes any existing in files 'foo.c', 'bar.ml' and 'bar.mli'. Files which do not have a header are kept unchanged. CONFIGURATION FILE
File types and format of header may be specified by a configuration file. By default, the default builtin configuration file given in figure 2 is used. You can also use your own configuration file thanks to the -c option : headache -c myconfig -h myheader foo.c bar.ml bar.mli In order to write your own configuration, you can follow the example given in figure 2. A configuration file consists in a list of entries separated by the character '|'. Each of them is made of two parts separated by an '->'. The first one is a regular expression. (Regular expression are enclosed within double quotes and have the same syntax as in Gnu Emacs.) headache determines file types according to file basenames; thus, each file is dealt with using the first line its name matches. The second one describes the format of headers for files of this type. It consists of the name of a model (e.g. 'frame'), possibly followed by a list of arguments. Arguments are named: 'open:"(*"' means that the value of the argument 'open' is '(*'. headache currently supports three models and a special keyword: frame With this model, headers are generated in a frame. This model requires three arguments: 'open' and 'close' (the opening and closing sequences for comments) and 'line' (the character used to make the horizontal lines of the frame). Two optional arguments may be used 'margin' (a string printed between the left and right side of the frame and the border, by default two spaces) and 'width' (the width of the inside of the frame, default is 68). lines Headers are typeset between two lines. Three arguments must be provided: 'open' and 'close' (the opening and closing sequences for comments), 'line' (the character used to make the horizontal lines). Three optional arguments are allowed: 'begin' (a string typeset at the beginning of each line, by default two spaces), 'last' (a string typeset at the beginning of the last line) and 'width' (the width of the lines, default is 70). no This model generates no header and has no argument. skip Skip line corresponding to one of the "match" parameters regexp. For this kind of line, every first part pattern that matches the file basename is taken into account. SEE ALSO
/usr/share/doc/headache/manual.html AUTHOR
This manual page was written by Sylvain Le Gall gildor@debian.org Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Lesser General Public License, Version 2.1 or any later version published by the Free Software Foundation; considering as source code all the file that enable the production of this manpage. AUTHOR
Sylvain Le Gall <gildor@debian.org> Author. COPYRIGHT
Copyright (C) 2003, 2004, 2005, 2006 Sylvain Le Gall [FIXME: source] Feb 15, 2004 HEADACHE(1)
All times are GMT -4. The time now is 09:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy