Parsing out the first (top) data lines of each category


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parsing out the first (top) data lines of each category
# 1  
Old 12-22-2010
Parsing out the first (top) data lines of each category

Hi All,
I need some help in parsing out the first (top) data lines of each category (categories are based on the first column a, b, c, d, e.( see example file below) from a big file
Code:
a dfg 3 6 8 9
a fgh 5 7 0 9
a gkl 5 2 4 7
a glo 7 0 1 5
b ghj 9 0 4 2
b mkl 7 8 0 5
b jkl 9 0 4 5
c jkl 2 4 5 7
c klm 0 3 6 7
d kml 9 2 5 7
e lsd 3 6 8 9
e jko 4 5 7 8
e kom 8 0 2 1

The output file I need is
Code:
a dfg 3 6 8 9
b ghj 9 0 4 2
c jkl 2 4 5 7
d kml 9 2 5 7
e lsd 3 6 8 9

I just wanted to parse out the first line in each category of a, b, c, d. In my original file I have thousands of such category.
Please let me know the best way in awk or sed to perform this task.
Thanks
# 2  
Old 12-22-2010
Code:
$ awk '!L[$1]++' file1
a dfg 3 6 8 9
b ghj 9 0 4 2
c jkl 2 4 5 7
d kml 9 2 5 7
e lsd 3 6 8 9

You could also do it with sort:
Code:
sort -uk1,1 file1


Last edited by Scott; 12-22-2010 at 12:51 PM..
This User Gave Thanks to Scott For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Inserting column data based on category assignment

please help with the following. I have 4 col data .. instrument , category, variable and value. the instruments belong to particular categories and they all measure some variables (var1 and var2 in this example), the last column is the value an instrument outputs for a variable. I have used... (0 Replies)
Discussion started by: ritakadm
0 Replies

2. Shell Programming and Scripting

Data filtering and category assigning

Please consider the following file, I have many groups which can be of 3 types, T1 (Serial_Number 1) T2 (Serial_Number 2) and T1*T2 (all other Serial_Number). I want to only consider groups that have both T1 and T2 present and their values are different from each other. In the example file,... (8 Replies)
Discussion started by: jianp83
8 Replies

3. Shell Programming and Scripting

Print n lines from top and n lines from bottom of all files with .log extenstion

Oracle Linux 6.4 In a directory I have more than 300 files with the extension .log I want the first 5 and last 5 lines of these .log files to be printed on screen with each file's name. Expected output : Printing first 5 and last 5 lines of FX_WT_Feb8_2014.log !! Authentication... (7 Replies)
Discussion started by: kraljic
7 Replies

4. Shell Programming and Scripting

Parsing XML (and insert data) then output data (bash / Solaris)

Hi folks I have a script I wrote that basically parses a bunch of config and xml files works out were to add in the new content then spits out the data into a new file. It all works - apart from the xml and config file format in the new file with XML files the original XML (that ends up in... (2 Replies)
Discussion started by: dfinch
2 Replies

5. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

6. Shell Programming and Scripting

Remove x lines form top and y lines form bottom using AWK?

How to remove x lines form top and y lines form bottom. This works, but like awk only cat file | head -n-y | awk 'NR>(x-1)' so remove last 3 lines and 5 firstcat file | head -n-3 | awk 'NR>4' (5 Replies)
Discussion started by: Jotne
5 Replies

7. Shell Programming and Scripting

Parsing of TOP output

Hi , i am trying to set up an alert, when CPU usage (0.2%us in below output) is more than 40% top | head | grep '^Cpu' Cpu(s): 0.2%us, 0.2%sy, 0.0%ni, 99.1%id, 0.6%wa, 0.0%hi, 0.0%si, 0.0%st using CUT, i pulled the value 0.2 and assigned to CPU (variable) CPU=$(expr `top | head -10... (5 Replies)
Discussion started by: Prateek007
5 Replies

8. Shell Programming and Scripting

Parsing the output from top

Guys can you help me fix this parse error. Here's my script. #!/bin/bash # Set up limit below NOTIFY="6.0% us 6.1% us 6.2% us 6.3% us 6.5% us 6.6% us 6.7% us 6.8% us 6.9% us 7.0% us" # CPU Usage every minute TOP="$(top -b -n2 -d 00.20 |grep Cpu|tail -1 | awk -F ":" '{ print $2 }' | cut... (3 Replies)
Discussion started by: redtred
3 Replies

9. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

10. UNIX for Dummies Questions & Answers

Collecting data from TOP to a file

Is there a way to collect data from top command to a file at regular intervals. I need this on HP-UX. I need to gather resident memory usage. I know that sar will give %cpu usage. There is a pmap command which gives memory usage in solaris. Is there a similar command in HP-UX Thanks (5 Replies)
Discussion started by: sssow
5 Replies
Login or Register to Ask a Question