search and grab data from a huge file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers search and grab data from a huge file
# 1  
Old 04-06-2006
Question search and grab data from a huge file

folks,

In my working directory, there a multiple large files which only contain one line in the file. The line is too long to use "grep", so any help?

For example, if I want to find if these files contain a string like "93849", what command I should use?

Also, there is oder_id number inside the line, just a part of the line, the format is : .....<order_id>92873</order_id>....., how can I grab all the order ids from these file?

Thanks and have a nice day!!! Smilie
# 2  
Old 06-06-2006
Try this

Hi, I had the same issue with a large XML file being on one line. If you have Pearl on your server, you can use this command to add carriage returns after the closing xml tags so you can use the sed command.

cat $file | perl -p -i -e 's@><@>\n<@g' > `echo ${file} | cut -d. -f1`_broken.xml

Replace "$file" with your filename. This will create a copy of the file called filename_broken.xml with carriage returns.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to grab data in range then search for pattern

im using the following code to grab data, but after the data in the range im specifying has been grabbed, i want to count how many instances of a particular pattern is found? awk 'BEGIN{count=0} /parmlib.*RSP/,/seqfiles.*SSD/ {print; count++ } /103 error in ata file/ END { print count }'... (3 Replies)
Discussion started by: SkySmart
3 Replies

2. Shell Programming and Scripting

How to grab a block of data in a file with repeating pattern?

I need to send email to receipient in each block of data in a file which has the sender address under TO and just send that block of data where it ends as COMPANY. I tried to work this out by getting line numbers of the string HELLO but unable to grab the next block of data to send the next... (5 Replies)
Discussion started by: loggedout
5 Replies

3. Shell Programming and Scripting

Grab data within a table in a long log file.

in my file which is a rather long log file it contains many text and tables and there is one table with 15 columns and I am interested to read in the value in column6 and its corresponding value in column2. Trouble is I do not know how to script it as the line number various between different log... (8 Replies)
Discussion started by: piynik
8 Replies

4. Shell Programming and Scripting

Grab 2 pieces of data within a file

I am a newbie and what I have is a captured file of content. I want to be able to grab 2 pieces of data, multiple times and print them to the screen. DataFile owner: locke user: fun data size: 60 location: Anaheim owner: david user: work data size: 80 location: Orange my script... (2 Replies)
Discussion started by: greglocke
2 Replies

5. Shell Programming and Scripting

Grab data between 2 keywords any do an array operation and write the file intact

Hi Unix Gurus, I need to grep for a block that is between a start and end keyword and then in between I need to find and replace a keyword. for eg: I need to search between Test = 000; and Test = 000; and find K9 and replace with M9 INPUT FILE Define { Replace = K9; Test =... (6 Replies)
Discussion started by: naveen@
6 Replies

6. Shell Programming and Scripting

search a number in very very huge amount of data

Hi, I have to search a number in a very long listing of files.the total size of the files in which I have to search is 10 Tera Bytes. How to search a number in such a huge amount of data effectively.I used fgrep but it is taking many hours to search. Is there any other feasible solution to... (3 Replies)
Discussion started by: vsachan
3 Replies

7. Shell Programming and Scripting

Optimised way for search & replace a value on one line in a very huge file (File Size is 24 GB).

Hi Experts, I had to edit (a particular value) in header line of a very huge file so for that i wanted to search & replace a particular value on a file which was of 24 GB in Size. I managed to do it but it took long time to complete. Can anyone please tell me how can we do it in a optimised... (7 Replies)
Discussion started by: manishkomar007
7 Replies

8. Shell Programming and Scripting

search a string in a huge file

How to search a string which has occured numerous times in a single row. I tried many options, I am facing issue with the file size. Anything I go for, it says it is huge.. File is 82MB. Assume, the file contains the string 'Name' in many places.. Something Like below. ... (5 Replies)
Discussion started by: Muthuraj K
5 Replies

9. Shell Programming and Scripting

insert a header in a huge data file without using an intermediate file

I have a file with data extracted, and need to insert a header with a constant string, say: H|PayerDataExtract if i use sed, i have to redirect the output to a seperate file like sed ' sed commands' ExtractDataFile.dat > ExtractDataFileWithHeader.dat the same is true for awk and... (10 Replies)
Discussion started by: deepaktanna
10 Replies

10. Shell Programming and Scripting

How to extract data from a huge file?

Hi, I have a huge file of bibliographic records in some standard format.I need a script to do some repeatable task as follows: 1. Needs to create folders as the strings starts with "item_*" from the input file 2. Create a file "contents" in each folders having "license.txt(tab... (5 Replies)
Discussion started by: srsahu75
5 Replies
Login or Register to Ask a Question