Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

AWK Script to Capture Each Line of File As Variable

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting AWK Script to Capture Each Line of File As Variable
# 1  
Old 06-28-2011
AWK Script to Capture Each Line of File As Variable
Hi All,

I'm working on creating a parts database. I currently have access to a manufacturer database in HTML and am working on moving all of the data into a MySQL db. I have created a sed script that strips out the HTML and unnecessary info and separates the script into one line for each field. Now, if each HTML file had the exact same number of fields, this would be a piece of cake, but the problem is that the manufacturer has omitted some fields where they are not applicable or do not have the data and some fields are more than one line, which makes my program a bit harder.

I'm still a newbie with awk, but have spent a few days studying the awk O'Reilly book. My goal is to get each line of the input file into a separate variable (or variable array) in awk so I can then program some logic operators to determine which data belongs in which database field.

I have written the below program which changes the field separator to "\n". I realize this doesn't work as awk is parsing each line one at a time until EOF. My question is how do I get this to work?
Code:
# nltest.awk - check to see if new line \n can be used as a separator
BEGIN { FS="\n" }
{
if ($1) print $1; 
if ($2) print $2; 
if ($3) print $3; 
if ($4) print $4;
part["manufacturer"] = $1;
part["model"] = $2;
part["part_number"] = $3;
part["serial_number"] = $4;

print "The manufacturer is " $part["manufacturer"]
print "The model is " $part["model"]
print "The part number is " $part["part_number"]
print "The serial number is" $part["serial_number"]
}

Thanks,

Dkr.
Last edited by Franklin52; 06-28-2011 at 03:45 AM.. Reason: Please use code tags for code and data samples, thank you
# 2  
Old 06-28-2011
It should be helpful if you post the input file and the desired output.

Please use code tags.
# 3  
Old 06-29-2011
Here's the input file to the awk script (some of the lines are truncated, but you can see the pattern here):

Code:
Acer Aspire 4552-5078 Specs (Turion II P540 2.4 GHz, 14" TFT) - Laptops -
* Manufacturer: Acer
* Part Number:LX.R6202.005
General
* Built-in Devices Wireless LAN antenna,
  Speaker
* Width 13.4 in
* Depth 10.4 in
* Height 1.3 in
* Weight 5.5 lbs
* Localization United States
* Notebook type Mid-size laptops (5-7 lbs.) ,
  Budget
 
* Processor AMD Turion II mobile processor P540 / 2.4 GHz
* Multi-Core Technology Dual-Core
* 64-bit Computing Yes
* Chipset Type AMD M880G
 
* Type L2 cache
* Installed Size 2 MB
 
* Installed Size 4 GB / 8 GB (max)
* Technology DDR3 SDRAM
* Form Factor SO DIMM 204-pin
* Configuration Features 2 x 2 GB
 
* Floppy Drive None
* Hard Drive 500 GB - 5400 rpm
* Storage Removable None
* Hard drive type Portable

I'm looking to get this in a CSV or tab-delimited format such as:

"Acer", "LX.R6202.005", "Wireless LAN antenna, Speaker", "13.4 in", "10.4 in", "1.3 in", "5.5 lbs", ...

In order to do this, I need to separate the lines into different variables and then determine which variables are on which line.

Dkr.

---------- Post updated 06-29-11 at 12:16 AM ---------- Previous update was 06-28-11 at 11:49 AM ----------

This looks like it's going to be difficult to do in AWK, so I've written a Perl script. I'm not done yet, but have gotten past some of the larger hurdles.

Dkr.
# 4  
Old 06-29-2011
Try to clear the file:
Code:
sed '/,$/{N; s/\n//}; /^*/!d ; s/^* //; s/ *,  */, /'

Then you need to make a list of keywords like this:
Manufacture
Part Number
Built-in Devices
Width
...
After that your task should be easy in any script language (bash, perl, awk)...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Capture the last record number using "awk" NR variable

Hi Team. I am trying to capture the last record number from a file using the below command ( assuming abc.txt has 21 records and I want 21 as output ) awk'{c=NR;print c}'abc.txt But it is printing all the record number. Can someone please help modify the above command? (8 Replies)
Discussion started by: chatwithsaurav
8 Replies

2. Shell Programming and Scripting

awk to lookup stored variable in file and print matching line

The bash bash below extracts the oldest folder from a directory and stores it in filename That result will match a line in bold in input. In the matching line there is an_xxx digit in italics that (once the leading zero is removed) will match a line in link. That is the lint to print in output.... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

How to read a two files, line by line in UNIX script and how to assign shell variable to awk ..?

Input are file and file1 file contains store.bal product.bal category.bal admin.bal file1 contains flip.store.bal ::FFFF:BADC:CD28,::FFFF:558E:11C5,6,8,2,1,::FFFF:81C8:CA8B,::FFFF:BADC:CD28,1,0,0,0,::FFFF:81C8:11C5,2,1,0,0,::FFFF:81DC:3111,1,0,1,0 store.bal.... (2 Replies)
Discussion started by: veeruasu
2 Replies

4. Shell Programming and Scripting

awk script file command line options

Being new to awk I have a really basic question. It just has to be in the archives but it didn't bite me when I went looking for it. I've written an awk script, placed it in a file, added the "#!/usr/bin/awk -f" at the top of the script and away I go. "% myAwk <inputfile>" gives me exactly what... (2 Replies)
Discussion started by: tomr2k
2 Replies

5. Shell Programming and Scripting

looking for help on script to capture file permission.

Hi Guys, I'm a DBA and need help on shell scripting. My Oracle Database is sitting on HP-UX machine. Anyone has a script that can spool out permission of all oracle binary files in the below directory: /opt/ora10g/oracle/ Format to be spooled out : chmod <exisiting permission> filename... (10 Replies)
Discussion started by: ciaciachew
10 Replies

6. Shell Programming and Scripting

awk script: print line number n of another file

Hi, I wrote an awk script to analyse file A. I call the script with files A and B. File A has lines like: 000000033100001 000000036100001 000000039100001 The first 9 characters are interpreted as a line number; for each line number found I want to output this line number of file B. ... (13 Replies)
Discussion started by: kpg
13 Replies

7. Shell Programming and Scripting

awk/shell script to print each line to a file

Dear People, My query is: have a file, which looks likes this: 10 20 30 40 50 1 2 3 4 5 100 200 300 400 500 what i need is: "PRINT EACH LINE TO AN UNIQUE FILE" desired output: file 1 10 20 30 40 50 file 2 1 2 3 4 5 (3 Replies)
Discussion started by: saint2006
3 Replies

8. Shell Programming and Scripting

Need awk script to add a prefix to each line in file

Hello , I have file with below content : '165567885', '165568443', '165568805', I need an awk script that would add a prefix zero after first ' . Like '0165567885', '0165568443', '0165568805', Please help. Thanks in advance. (5 Replies)
Discussion started by: rmv
5 Replies

9. Shell Programming and Scripting

Capture first N Bytes from first line in a file

Hi Guyz, I need to capture first N Bytes from the first line of my file. Eg. If i have following data in File1 414d51204541495052475731202020204a910846230e420c Hello 3621363663212 Help Required Then, i want the value of first 48 Bytes to be stored in a variable. That is, variable... (5 Replies)
Discussion started by: DTechBuddy
5 Replies

10. Shell Programming and Scripting

print any required line by its line no using awk and its NR variable

how to print any required line by its line no using awk and its NR variable for eg: ------------ 121343 adfdafd 21213sds dafadfe432 adf.adf%adf --------------- requied o/p if give num=3 it print: 21213sds -------------------------------------- (2 Replies)
Discussion started by: RahulJoshi
2 Replies
Login or Register to Ask a Question