How do I use grep to pull incremental data and send to multiple files?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How do I use grep to pull incremental data and send to multiple files?
# 1  
Old 07-08-2015
How do I use grep to pull incremental data and send to multiple files?

Hi Everyone,
Im currently using the below code to pull data from a large CSV file and put it into smaller files with just the data associated with the number that I "grep".

Code:
grep 'M053' test.csv > test053.csv

Is there a way that I can use grep to run through my file like the example below and take each line that has a match for the first 4 digits and write the matches to a corresponding file? I can use the above command to get this done but I want to know of a way to run a command or script that will process the whole file at one time. So given the below data I would end up with 5 different files IE test051.csv test053.csv, test103.csv, test105.csv and test115.csv also these numbers will change from 051 to 999. Thanks in advance

Code:
M051POD071,
M053POD071,
M103POD075,
M105POD073,
M105POD075,
M115PAD011,
M115PAD012,
M115PAD024,

# 2  
Old 07-08-2015
Try
Code:
awk '{print >("test" substr($1,2,3) ".csv")}' file

This User Gave Thanks to RudiC For This Post:
# 3  
Old 07-08-2015
Ok that totally work can you explain how that worked? I did not have any idea that awk would have done that.

Thank you!!!!
# 4  
Old 07-08-2015
It creates the requested output file name for the redirection from two constants and the 3 chars (2,3,4) from your data line.
This User Gave Thanks to RudiC For This Post:
# 5  
Old 07-08-2015
Thank you again!

Once last question as I'm sure they will change the data on me again at some point. How would I compensate for a change to the data where they leave off the "M"? like in the below data?

Code:
M051POD071, M053POD071, M103POD075, 105POD073, M105POD075, M115PAD011, 115PAD012, M115PAD024,

# 6  
Old 07-08-2015
Try
Code:
awk '{X=substr($1,1,4); gsub (/[A-Za-z]/,"",X); print > ("test" X ".csv")}' file

This User Gave Thanks to RudiC For This Post:
# 7  
Old 07-09-2015
Once again thank you! That worked great.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

In PErl script: need to read the data one file and generate multiple files based on the data

We have the data looks like below in a log file. I want to generat files based on the string between two hash(#) symbol like below Source: #ext1#test1.tale2 drop #ext1#test11.tale21 drop #ext1#test123.tale21 drop #ext2#test1.tale21 drop #ext2#test12.tale21 drop #ext3#test11.tale21 drop... (5 Replies)
Discussion started by: Sanjeev G
5 Replies

2. Shell Programming and Scripting

Grep strings on multiple files and output to multiple files

Hi All, I want to use egrep on multiple files and the results should be output to multiple files. I am using the below code in my shell script(working in Ksh shell). However with this code I am not attaining the desired results. #!/bin/ksh ( a="/path/file1" b="path/file2" for file in... (4 Replies)
Discussion started by: am24
4 Replies

3. UNIX for Dummies Questions & Answers

Pull out multiple lines with grep patternfile

Hi, I'm trying to get lines from a file using identifiers in the first two columns. I have used: cat MasterFile.txt | grep -f Pattern.txt and the lines I want display on screen. If I try to put them in a file the file is created but stays empty: cat MasterFile.txt | grep -f Pattern.txt... (14 Replies)
Discussion started by: FGPonce
14 Replies

4. Shell Programming and Scripting

Grep multiple instances and send email.

Removed (15 Replies)
Discussion started by: saisneha
15 Replies

5. UNIX for Dummies Questions & Answers

Grep multiple strings in multiple files using single command

Hi, I will use below command for grep single string ("osuser" is search string) ex: find . -type f | xarg grep -il osuser but i have one more string "v$session" here i want to grep in which file these two strings are present. any help is appreciated, Thanks in advance. Gagan (2 Replies)
Discussion started by: gagan4599
2 Replies

6. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is... (3 Replies)
Discussion started by: Liverpaul09
3 Replies

7. Shell Programming and Scripting

How do you use pull data from multiple lines to do a for statement?

Guys I am having a problem with being able to find missing monitors in a configuration check script I am trying to create for accountability purposes for managing a large number of systems. What I am trying to do is run a script that will look at the raw config data in a file and pull all the pool... (7 Replies)
Discussion started by: scottzx7rr
7 Replies

8. Shell Programming and Scripting

Cat 114 files using grep or awk to pull muliple fields

Files xxxxxxx.txt ------------------------------------------------------------------------------------------------------------------------------------ Req.By: xxxxxxx WABUSH MINES - xxxxxx MINE (1001) Page: 1 Run on: 12/14/09... (4 Replies)
Discussion started by: sctxms
4 Replies

9. Shell Programming and Scripting

How to Pull out multiple files from DB table and redirect all those files to a differetn directory?

Hi everyone!! I have a database table, which has file_name as one of its fields. Example: File_ID File_Name Directory Size 0001 UNO_1232 /apps/opt 234 0002 UNO_1234 /apps/opt 788 0003 UNO_1235 /apps/opt 897 0004 UNO_1236 /apps/opt 568 I have to... (3 Replies)
Discussion started by: ss3944
3 Replies

10. Shell Programming and Scripting

Need to send multiple files during ftp

Hi guys... I'm working on #!/bin/sh script in a Solaris 7 box that should send several files over to another machine via FTP. Here's what the script looks like: # This script will send the daily MSP customer counts # to the Crystal Reports server located at 192.168.2.106 cd... (2 Replies)
Discussion started by: cdunavent
2 Replies
Login or Register to Ask a Question