Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Parsing through a file with awk/sed

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Parsing through a file with awk/sed
# 1  
Old 12-14-2010
Parsing through a file with awk/sed
I don't necessary have a problem, as I have a solution. It is just that there may be a better solution.

GOAL: Part one: Parse data from a file using the "\" as a delimiter and extracting only the last delimiter. Part two: Parse same file and extract everything but the last delimited item.

Background: I was given 600+ registry keys that needed to be queried. I was given a file with the concatenated keys and values. IE...HKLM\Software\Microsoft\Driver Signing\Policy. I need the value "Policy" separated from the rest of the key.

Sample data:
Code:
HKLM\System\CurrentControlSet\Control\Lsa\LimitBlankPasswordUse
Password Policy security settings are not registry keys.
HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\DontDisplayLastUserName
HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\DisableCAD
[snip]......

My Solution:
Part one of the problem was a snap to solve, I ran the following command and got all the values.
Code:
awk --field-separator='\' '{ print $NF }' regkeydump

OUTPUT:
Code:
LimitBlankPasswordUse

DontDisplayLastUserName
DisableCAD
[snip]......

Perfect, works like a charm. Part two is where problems begin. No matter what I try, I cannot get this command to recognize the "\"
Code:
awk 'BEGIN {FS=ORS="\"} {for (i=1;i<NF;i++) print $i}'

So as a work around I replaced the "\" with ":" in my data file
Code:
cat regkeydump | tr '\' ':' > regkeydumpprep

Then ran the awk command
Code:
awk 'BEGIN {FS=ORS=":"} {for (i=1;i<NF;i++) print $i}' regkeydumpprep |sed 's/$/\n/' > regkeysonly

OUTPUT:
Code:
HKLM:System:CurrentControlSet:Control:Lsa:HKLM:Software:Microsoft:Windows:CurrentVersion:Policies:HKLM:Software:Microsoft:Windows:CurrentVersion:Policies:System
[snip]......

Some problems arise, the sed command is not working properly creating newlines. Second it removed a line that did not have the delimiter in the line. This isn't a huge deal, but means that I now have to manually line up the data. I now run a new sed command to replace the :HKLM with \nHKLM
Code:
sed -r "s/:HKLM/\\`echo -e '\nHKLM'`/g" regkeysonly > regkeysclean

OUTPUT:
Code:
HKLM:System:CurrentControlSet:Control:Lsa
HKLM:Software:Microsoft:Windows:CurrentVersion:Policies
HKLM:Software:Microsoft:Windows:CurrentVersion:Policies:System
[snip]......

I now replace the ":" with the "\" to get the data back to its original state
Code:
cat regkeysclean | tr ':' '\' > finished

OUTPUT:
Code:
HKLM\System\CurrentControlSet\Control\Lsa
HKLM\Software\Microsoft\Windows\CurrentVersion\Policies
HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System
[snip]......
`

This works, but I think that if I could get the awk loop to use the "\" as the delimiter and the sed command to work this would be a two liner. Any thoughts?
Last edited by Scott; 12-14-2010 at 11:43 AM.. Reason: Please use code tags
# 2  
Old 12-14-2010
Code:
sed 's:\\[^\\]*$::' yourfile

This User Gave Thanks to ctsgnb For This Post:
OrangeYaGlad
# 3  
Old 12-14-2010
Code:
cat inputFile | while read line
do
   firstPart=$(echo $line | sed 's/\(.*\)\\\(.*\)/\1/')
   secondPart=$(echo $line | sed 's/\(.*\)\\\(.*\)/\2/')
   echo $firstPart
   echo $secondPart
done

OR
Code:
cat inputFile | while read line
do
   firstPart=${line%\\*}
   secondPart=${line##*\\}
   echo $firstPart
   echo $secondPart
done

Last edited by anurag.singh; 12-14-2010 at 11:21 AM..
# 4  
Old 12-14-2010
Code:
awk NF-=1 FS=\\ OFS=\\ file

This User Gave Thanks to Scrutinizer For This Post:
OrangeYaGlad
# 5  
Old 12-14-2010
if you are in solaris FS="\\" will only work with this awk "/usr/xpg4/bin/awk" and will not work with nawk , in linux system it will work with gawk:-


Code:
/usr/xpg4/bin/awk 'NF>1{$NF=""}1' FS="\\"  OFS="\\" infile.txt

Code:
O/P:-
HKLM\System\CurrentControlSet\Control\Lsa\
Password Policy security settings are not registry keys.
HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\
HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\

# 6  
Old 12-14-2010
re: Parsing through a file with awk/sed
Thank you both ctsgnb and Scrutinizer.

Both commands work and are a nice one liner.

The only difference is that that ctsgnb's sed one liner does not parse out the lines without the delimiter. Which means a little less manual work for me.

I am more familiar with awk and understand exactly what it is doing but will need to to a little research on the sed command to get a full understanding. Unfortunately I don't get to use *nix as often as I would like, but now I know to use \\ to ensure the backslash is recognized.
# 7  
Old 12-14-2010
did you try my code , I am afraid that I understand you wrongly , did I?

what is the o/p you need to get?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Interesting awk/Perl/sed parsing challenge

I have a log with entries like: out/target/product/imx53_smd/obj/STATIC_LIBRARIES/libwebcore_intermediates/Source/WebCore/bindings/V8HTMLVideoElement.cpp : target thumb C++: libwebcore <=... (8 Replies)
Discussion started by: glev2005
8 Replies

2. Shell Programming and Scripting

awk/sed line parsing

I'm new to shell programming, but I think I learn best by following an example. I'm trying to cook up an awk/sed script, but I obviously lack the required syntax skills to achieve it. The output that I get from running my ksh script looks like this: I need to search each numbered line for... (10 Replies)
Discussion started by: iskatel
10 Replies

3. Shell Programming and Scripting

Parsing with awk or sed

I want to delete corrupt records from a file through awk or sed. Can anyone help me with this Thanks Striker Change subject to a descriptive one, ty. (1 Reply)
Discussion started by: Rahul_us
1 Replies

4. Shell Programming and Scripting

Another parsing line awk or sed problem

Hi, After looking on different forums, I'm still in trouble to parse a parameters line received in KSH. $* is equal to "/AAA:111 /BBB:222 /CCC:333 /DDD:444" I would like to parse it and be able to access anyone from his name in my KSH after. like echo myArray => display 111 ... (1 Reply)
Discussion started by: RickTrader
1 Replies

5. Shell Programming and Scripting

Line Parsing using sed and awk

Hi Guys, I need help with processing data in a file, line by line. My file test.txt has X_Building_X5946/X0 BUT/U_msp/RdBuMon_d2_B_00 BUT/U_msp/FfRmDaMix_d2_Pi3 Test_Long xp=849.416 yp=245.82 xn=849.488 yn=245.82 w=0.476 l=0.072 fault_layer="Al_T01_Mod" $ $X=849416 $Y=245582... (2 Replies)
Discussion started by: naveen@
2 Replies

6. Shell Programming and Scripting

String parsing with awk/sed/?

If I have a string that has some name followed by an ID#(ex.B123456) followed by some more #'s and/or letters, would it be possible to just grab the ID portion of this string? If so how? I am pretty new with these text tools so any help is appreciated. Example: "Name_One-B123456A-12348A" (2 Replies)
Discussion started by: airon23bball
2 Replies

7. Shell Programming and Scripting

awk/sed for parsing file

Hi All, I have a log file like this E Mon Oct 06 00:17:08 2008 xxx2 cm:10614 fm_pi2_svc_iptv_purchase.c:149 1:pin_deferred_act:10601:11:169:1223245028:16 pi2_op_svc_iptv_purchase error <location=PIN_ERRLOC_FM:5 class=PIN_ERRCLASS_SYSTEM_DETERMINATE:1... (10 Replies)
Discussion started by: subin_bala
10 Replies

8. Shell Programming and Scripting

Parsing a file (sed/awk?)

Hello people, newbie question. I'm trying to parse these type of file 1 "CAR " " C1 " " " 6 0 C1 2 "CAR " " O1A" " " 8 0 O1A 3 "CAR " " O1B" " " 8 -1 O1B 4 "CAR " " C2 " " " 6 0 C2 5 "CAR " " C3 " " " 6 ... (10 Replies)
Discussion started by: aristegui
10 Replies

9. Shell Programming and Scripting

parsing xml with awk/sed

Hi people!, I need extract from the file (test-file.txt) the values between <context> and </context> tag's , the total are 7 lines,but i can only get 5 or 2 lines!!:confused: Please look my code: #awk '/context/{flag=1} /\/context/{flag=0} !/context/{ if (flag==1) p rint $0; }'... (3 Replies)
Discussion started by: ricgamch
3 Replies

10. Shell Programming and Scripting

awk sed parsing

hi , i would like to parse some file with the fallowing data : data data data "unwanted data" data data "unwanted data" data data data data #unwanted data. what i want it to have any coments between "" and after # to be erased using awk or/and sed. has anyone an idea? thanks. (3 Replies)
Discussion started by: Darsh
3 Replies
Login or Register to Ask a Question