Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sed filtering issue

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sed filtering issue
# 1  
Old 10-29-2008
Sed filtering issue
The problem I have is that I have 23,000 records I need to sort through to pull out LEN: XXXX XX XX XX XX and NCOS: XXX entries from so I can insert them into a database. But some of my records include TYPE: ISDN, THE DN IS UNASSIGNED, or INVALID entries in between some records and I would like them to print where they are supposed to. Here is a sample of my records:
Code:
LEN:     HOST  11 0 14 10
TYPE: SINGLE PARTY LINE
SNPA: 315
DIRECTORY NUMBER:     6320171
LINE CLASS CODE:  IBN
IBN TYPE: STATION
CUSTGRP:           KAFB     SUBGRP: 0  NCOS: 16
SIGNALLING TYPE:  DIGITONE
CARDCODE:  6X17AC    GND: N  PADGRP: ONS  BNV: NL MNO: N
PM NODE NUMBER     :    72
PM TERMINAL NUMBER :    459
OPTIONS:
3WC CLF RAG AVT PREMTBL DGT CND NOAMA CNAMD NOAMA NAME PUBLIC BLDG 3135
FUELS SPB 5406320171 CFI $ I
-------------------------------------------------------------------------------
INVALID LEN SPECIFIED FOR THE QLEN COMMAND.
INVALID LEN SPECIFIED FOR THE QLEN COMMAND.
INVALID LEN SPECIFIED FOR THE QLEN COMMAND.
THE DN IS UNASSIGNED
-------------------------------------------------------------------------------
LEN:     HOST  13 1 16 22
TYPE: SINGLE PARTY LINE
SNPA: 315
DIRECTORY NUMBER:     6320063
LINE CLASS CODE:  IBN
IBN TYPE: STATION
CUSTGRP:           KAFB     SUBGRP: 0  NCOS: 51
SIGNALLING TYPE:  DIGITONE
CARDCODE:  6X17AC    GND: N  PADGRP: ONS  BNV: NL MNO: N
PM NODE NUMBER     :    77
PM TERMINAL NUMBER :    535
OPTIONS:
3WC COD CLF RAG LNR AVT PREMTBL DGT CND NOAMA CNAMD NOAMA NAME PUBLIC BLDN
220 SPB 5056320063 CPU 0 HOST 13 1 16 22 CFI 6344266 A
-------------------------------------------------------------------------------
INVALID LEN SPECIFIED FOR THE QLEN COMMAND.
THE DN IS UNASSIGNED
INVALID LEN SPECIFIED FOR THE QLEN COMMAND.
-------------------------------------------------------------------------------
LEN:     RST3  01 0 13 14
TYPE: SINGLE PARTY LINE
SNPA: 315
DIRECTORY NUMBER:     6327103
LINE CLASS CODE:  IBN
IBN TYPE: STATION
CUSTGRP:           KAFB     SUBGRP: 0  NCOS: 13
SIGNALLING TYPE:  DIGITONE
CARDCODE:  6X17BA    GND: N  PADGRP: ONS  BNV: NL MNO: N
PM NODE NUMBER     :    107
PM TERMINAL NUMBER :    431
OPTIONS:
DTM CWT 3WC COD CLF RAG CWI LNR DGT NAME PUBLIC 7149 RM233 CFI $ I
-------------------------------------------------------------------------------
INVALID LEN SPECIFIED FOR THE QLEN COMMAND.
-------------------------------------------------------------------------------

I was using this script to do just that, but I cannot get NCOS: XXX to show up:
Code:
#/bin/bash
sed -nr "/LEN:|TYPE:\ ISDN|THE\ DN\ IS\ UNASSIGNED|INVALID/s/.*(HOST.{12}|RST1.{12}|RST2.{12}|RST3.{12}|RST4.{12}|ISDN|UNASSIGNED|INVALID).*/\1/p" FILE > NEWFILE

This script works great for pulling the LEN: XXXX XX XX XX XX and NCOS: XXX from my records.(btw I need ncos and len to be on the same line)

Code:
#!/bin/bash
sed -nr '/^LEN:/,/--------/{                           
                            /LEN:|CUST/s/.*(HOST.{12}|RST1.{12}|RST2.{12}|RST3.{12}|RST4.{12}|NCOS: [0-9][0-9]*).*/\1/p
                           }' FILE | sed -e '$!N' -e 's/\n/  /g' > NEW FILE

Any help would be greatly appreciated by my coworkers and myself considering this project is due Monday and we are only about 20% done. Thanks.
# 2  
Old 10-30-2008
Can you give us an example of the output you expect from that sample input? Also you mention TYPE: ISDN records, but didn't give them in the examples.
# 3  
Old 10-31-2008
Since it's due on Monday you've probably found a workaround by now, but I'll take a punt anyway and you can probably modify it to your requirements:

Code:
awk -F': *' '
        /LEN:/ { len=$2 }
        /NCOS:/ { ncos=$NF }
        /----/ { print len,ncos }
        /INVALID|UNASSIGNED/
' inputfile

Gives:

Code:
HOST  11 0 14 10 16
INVALID LEN SPECIFIED FOR THE QLEN COMMAND.
INVALID LEN SPECIFIED FOR THE QLEN COMMAND.
INVALID LEN SPECIFIED FOR THE QLEN COMMAND.
THE DN IS UNASSIGNED
HOST  11 0 14 10 16
HOST  13 1 16 22 51
INVALID LEN SPECIFIED FOR THE QLEN COMMAND.
THE DN IS UNASSIGNED
INVALID LEN SPECIFIED FOR THE QLEN COMMAND.
HOST  13 1 16 22 51
RST3  01 0 13 14 13
INVALID LEN SPECIFIED FOR THE QLEN COMMAND.
RST3  01 0 13 14 13

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File filtering using awk or sed

Hello Members, I have a file, having below contents: <KEYVALUE>$4,0,1,4,966505098999--&gt;RemoteSPC: 13 SSN: 146</KEYVALUE> <KEYVALUE>$4,123,1,4,966505050198--&gt;RemoteSPC: 1002 SSN: 222,Sec:RemoteSPC: 1004 SSN: 222</KEYVALUE> <KEYVALUE>$4,123,1,4,966505050598--&gt;RemoteSPC: 1002 SSN:... (9 Replies)
Discussion started by: umarsatti
9 Replies

2. Shell Programming and Scripting

Filtering data using uniq and sed

Hello, Does anyone know an easy way to filter this type of file? I want to get everything that has score (column 2) 100.00 and get rid of duplicates (for example gi|332198263|gb|EGK18963.1| below), so I guess uniq can be used for this? gi|3379182634|gb|EGK18561.1| 100.00... (6 Replies)
Discussion started by: narachaid
6 Replies

3. Shell Programming and Scripting

Awk/sed : help on:Filtering multiple lines to one:

Experts Good day, I want to filter multiple lines of same error of same day , to only 1 error of each day, the first line from the log. Here is the file: May 26 11:29:19 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out) May 26 11:29:19 cmihpx02 vmunix: NFS... (4 Replies)
Discussion started by: rveri
4 Replies

4. Shell Programming and Scripting

sed filtering lines by range fails 1-line-ranges

The following is part of a larger project and sed is (right now) a given. I am working on a recursive Korn shell function to "peel off" XML tags from a larger text. Just for context i will show the complete function (not working right now) here: function pGetXML { typeset chTag="$1" typeset... (5 Replies)
Discussion started by: bakunin
5 Replies

5. UNIX for Dummies Questions & Answers

Grep filtering issue

Hi, I am on uname -a HP-UX mymachine B.11.31 U ia64 3223107173 unlimited-user license ps -exx| grep java | grep -i "GrafiteEventsInterfaces*" 19955 ? 55:22 /opt/app/app1/jdk150_07/bin/IA64N/java -server -Xms1024m -Xmx1024m -XX:PermSize=256m -XX:MaxPermSize=512m... (2 Replies)
Discussion started by: mohtashims
2 Replies

6. Shell Programming and Scripting

Filtering Issues Using sed and awk

Hi, I am currently using the sed and awk commands to filter a file that has multiple sets of data in different columns. An example of part of the file i am filtering is as follows; Sat Oct 2 07:42:45 2010 01:33:46 R1_CAR_12.34 Sun Oct 3 13:09:53 2010 00:02:34 R2_BUS_56.78 Sun... (4 Replies)
Discussion started by: crunchie
4 Replies

7. Shell Programming and Scripting

Issue Filtering Tail

Hi Folks, I have a log that contains data as shown below: 11:59:43,144 (1,850) Signal : .... 11:59:44,109 (1850) Bps : ..... I wish to remove "" from all lines and it is on the start of every line. I have achieved that successfully using the command: tail -f imp.log | sed 's/\... (4 Replies)
Discussion started by: umairrahman
4 Replies

8. Shell Programming and Scripting

Report filtering - Weird issue and interesting - UrgentPlease

Hi, Could any one help me to extract data from a report. I would like to get the two lines which are just below the separations I have a report like this -------------------------------------------------------------------------- Pid Command Inuse Pin Pgsp Virtual... (2 Replies)
Discussion started by: ajilesh
2 Replies

9. UNIX for Dummies Questions & Answers

filtering and copying contains of a file using awk/sed

Hello folks, I have 2 files one( file1) contains the ddl for a view and file 2 contains the view defination/alias columns. I want to merge the 2 into a third file using awk/sed as follows: cheers ! :b: FILE1 ----- PROMPT FIRST_VIEW CREATE OR REPLACE FORCE VIEW FIRST_VIEW AS SELECT... (2 Replies)
Discussion started by: jville
2 Replies

10. Shell Programming and Scripting

awk and sed filtering

Goo afternoon Sir'sould like to ask your help reagrding in this scenario using sed and awk. ******************************************************** Host:CDRMSAPPS1 Operating System:Linux 2.6.9-42.ELsmp Machine Type:UNIX Host Type:Client Version:5.1... (2 Replies)
Discussion started by: invinzin21
2 Replies
Login or Register to Ask a Question