sed or awk to convert text files with recurring headings to rows and colum


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sed or awk to convert text files with recurring headings to rows and colum
# 1  
Old 04-14-2008
sed or awk to convert text files with recurring headings to rows and colum

I have many text file reports generated by a Information Assurance tool that I need to get into a .CSV format or Excel tab delimited format. I want to use sed or awk to grab all the information in the sample text file below and create column headings:Risk ID, Risk Level, Category, Description, How To Fix, Affected Machines, Then I want to populate the cells(or fields?) with the data seen below. I think I can do this by grabbing everything between 2 of the heading strings, e.g: grab everything between Risk Level & Category: and add that to Risk Level field, then grab everything between Category and Description and add to Category value in Category column.

Problem ID: 3454
Risk Level: Low
Category: Accounts
Description: Local user account does not require a password.
How To Fix: To disable the allowance of blank password, change the following registry key: HKLM:System\CurrentControlSet\Control\Lsa\LimitBlankPasswordUse Set value to DWORD=1
Affected Machines: Student001 student0057 student6745.

Problem ID: 433
Risk Level: Med
Category: DoS
Description: Microsoft SynAttackProtect
How To Fix: Set: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\SynAttackProtect registry value to 1 (better protection) or 2 (best protection).
Affected Machines: Student021 student0032 student67.
Problem ID: 3454

Risk Level: High
Category: Miscellaneous
Description: Apple QuickTime 7.1.6 Update - Windows
How To Fix: Update to version 7.1.6 or later of QuickTime.
Affected Machines: Student011 student00907 student45.

Here is where I need to be:
Image
# 2  
Old 04-14-2008
input:
Code:
name:leo
age:28

name:stt
age:24


code:

Code:
awk 'BEGIN{FS=":";n=1}
{
if ($1=="name")
	name[n]=$2
else
{
	age[n]=$2
	n=n+1
}
}
END{
print "NAME    AGE"
for (i=1;i<=n;i++)
print name[i]"   "age[i]
}' a


output:

Code:
NAME    AGE
leo   28

stt   24

# 3  
Old 04-15-2008
I can't use colon: as FS because all the 'Descriptions' and 'How to Fix:' fields will be chock full of colons. Unless I somehow change the FS's to something else eccentric first e.g: do a find/replace on 'name:' to 'name*' change 'age:' to 'age*' then use * as FS. This is why I wanted to populate Category: field with all data between Category: and Description: strings, and same for other fields.
# 4  
Old 04-15-2008
assuming all the fields appear in the SAME order for different records/blocks....
not sure if you wanted ONE output .csv file or ONE csv PER your input file (as you may have many) - this assumes ONE output file for ALL the input files.

nawk -f bj.awk myInputFile.txt > myReport.csv

bj.awk:
Code:
BEGIN {
  FS=RS=""
  qq=sprintf("%c", "\"")
}
NR==1{
  for(i=1; i<=NF; i++)
     printf("%c%s%c%c", qq, substr($i, 1, index($i, ":")-1), qq, (i==NF) ? "\n" : ",")
}
{
  for(i=1; i<=NF; i++)
     printf("%c%s%c%c", qq, substr($i, index($i, ":")+2), qq, (i==NF) ? "\n" : ",")
}

# 5  
Old 05-07-2008
Quote:
Originally Posted by summer_cherry
input:
Code:
name:leo
age:28

name:stt
age:24


code:

Code:
awk 'BEGIN{FS=":";n=1}
{
if ($1=="name")
	name[n]=$2
else
{
	age[n]=$2
	n=n+1
}
}
END{
print "NAME    AGE"
for (i=1;i<=n;i++)
print name[i]"   "age[i]
}' a


output:

Code:
NAME    AGE
leo   28

stt   24

suppose if I have more than 2 fileds like name,age,total,avg then how the script changes ?
# 6  
Old 05-07-2008
With this script I got the output.

Scipt:
------

awk '
/siteid/ { printf $3 " " };
/spc/ {printf $3 " "} ;
/rset/ {printf $3 "\n" };
' log


Output:
-------
HYD 100 del
DEL 200 hyd
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Help with awk or sed Command to Replace Text in Files

Hello Everyone, I have many files like so: file1.txt file2.txt file3.txt Within each file I have many lines of random text separated by commas like so: abcAAA,123,defAA,456777,ghiA,789 jklB,101,mnoBBB,11211,pqrB,13111 stuCC,415,vwxCCCC,161,yzaC,718 I am trying to use SED or AWK to... (4 Replies)
Discussion started by: D3U5X
4 Replies

2. Shell Programming and Scripting

Convert rows into columns and create table with awk

Hello I've four fields . They are First Name, Last Name, Age, Country. So when I run a Unix command, I get below output with these fields comes every time in different order as you can see. Some times first name is the first row and other time last name is first row in the output and etc etc..... (9 Replies)
Discussion started by: rprpr
9 Replies

3. Shell Programming and Scripting

awk or sed? rows text to co

Hello Friends! I would like to help the masters ... I have a file with the entry below and would like a script for that output: Input file: 001 1 01-20152142711532-24S 1637909825/05/2015BAHIA SERVICOS R F, ... (1 Reply)
Discussion started by: He2
1 Replies

4. UNIX for Dummies Questions & Answers

Awk: convert rows to columns every n lines

Hi guys! I use AWK commands under GAMS to predispose the data files to be read by GAMS. I have a file which contains groups of data I need. Unfortunately I have the data spread in 3 rows for each subject. Here's an example (the file is really long) 1 0 2.0956 100.00 250.00 100.00 2.0956... (4 Replies)
Discussion started by: Pintug
4 Replies

5. Shell Programming and Scripting

Convert rows into columns using awk or perl

hi friends, i am able to parse cvs diff file using bit of cut and grep commands to produce following output in text file '''cvs-diff.txt''' Package-Name = dev-freetype. Old-Version = 2.4.8 New-Version = 2.4.10 Patches-removed = freetype-2.4.8-cross-compile.patch... (2 Replies)
Discussion started by: alexzander18
2 Replies

6. UNIX for Dummies Questions & Answers

Convert rows to columns using AWK

Hi , I am struck while coding AWK script. Need your help to convert rows into columns. I should copy only those rows which are marked to Y in a file and ignore N rows. Please help me find a solution. input file 1|abc|Y 2|cdf|Y 3|efg|N 4|xyz|Y my output should be something like this... (2 Replies)
Discussion started by: rashmisb
2 Replies

7. UNIX for Dummies Questions & Answers

joining files with different rows and colum

Hi all, I have two files: file one (9 rows, 3 columns): A 1 x1 B 2 f1 C 3 f3 D 4 u5 E 5 l9 F 6 h6 G 7 g4 H 8 r4 I 9 s2 file two (4 rows, 1 column): A B (2 Replies)
Discussion started by: anjas
2 Replies

8. Shell Programming and Scripting

get text between two special rows ?(awk or sed)?

Hello, I've the follwing text: gfdgfg -------------------------------- dfgfdgdfg fdgfdgfdgdgf fdgf ------------------------------ f g gdgf a constant string that i know --------------------------------------------- data I want to have data I want to have data I want to have data I... (16 Replies)
Discussion started by: eric_
16 Replies

9. Shell Programming and Scripting

Using sed (or similar) to rename variable headings

Hello, I'm rather new to the world of regular expressions and sed, though am excited by its possibilities. I have a particular task I'd like to achieve, and have googled the topic quite a bit. However, having found some codes that perform a task very similar to what I'd like to do, I can't for... (2 Replies)
Discussion started by: redseventyseven
2 Replies

10. Shell Programming and Scripting

Get text between two special rows ?( using awk or sed)?

Hi Friends I am facing some problem in extract the lines between two fixed lines for examplemy text file look like ... -------- line 1 line 2 line 3 --------- line 4 line 5 -------- line 6 line 7 line 8 line 9 line 10 --------- now i want the data between "-------" these... (4 Replies)
Discussion started by: sushantnirwan
4 Replies
Login or Register to Ask a Question