Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Splitting Files with awk into other directory

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Splitting Files with awk into other directory
# 1  
Old 01-23-2014
Splitting Files with awk into other directory
I am trying to split into different files using awk:

Code:
cat files | gawk '$1 ~ /---/ || $1 ~ /^deleting$/ || $1 ~ /^sorting$/ || $1 ~ /==/ {print}'| gawk '$1 ~ /---/ || $1 ~ /^deleting$/ || $1 ~ /^sorting$/ || $1 ~ /==/ {print}' |gawk '/[TEST]/{x="F"++i;}{print > x;}'

What I am trying to do is make F* files in the /tmp directory. The command works fine, but the problem is that it only splits the files into F1 F2 etc. into the current directory. I've searched but cannot find a way to put them in a directory other than the present directory, though it should be simple. Anyone have an idea? I would think "/tmp/x" but this didn't work and I've tried variations.
# 2  
Old 01-23-2014
Code:
... { print > "/path/to/" X }

You've got two of the exact same awk program in a row, you probably only ned one. And you can simplify that one a whole lot, too, and remove the cat -- awk does not need cat's help to read multiple files.


Code:
gawk '$1 ~ /---|(^deleting$)|(^sorting$)|==/' files | gawk '/[TEST]/{ x="F"++i;} {print > "/path/to/" X }'

Last edited by Corona688; 01-23-2014 at 12:53 PM..
This User Gave Thanks to Corona688 For This Post:
newbie2010
# 3  
Old 01-23-2014
@corona you missed something : small x
my opinion it's good if you close file,you might face problem if many files are opened
Code:
/[TEST]/{if(x){close(x)} x="F"++i}

This User Gave Thanks to Akshay Hegde For This Post:
Corona688
# 4  
Old 01-23-2014
Why not
Code:
awk '$1 ~ /---|(^(dele|sor)ting$)|==/ {if (/[TEST]/) {if(x) close(x) x="F"++i}; print > "/path/to/" x}' files

This User Gave Thanks to RudiC For This Post:
Corona688
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies

2. Shell Programming and Scripting

Splitting a text file into smaller files with awk, how to create a different name for each new file

Hello, I have some large text files that look like, putrescine Mrv1583 01041713302D 6 5 0 0 0 0 999 V2000 2.0928 -0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 5.6650 0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 3.5217 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

3. Open Source

Splitting files using awk and reading filename value from input data

I have a process that requires me to read data from huge log files and find the most recent entry on a per-user basis. The number of users may fluctuate wildly month to month, so I can't code for it with names or a set number of variables to capture the data, and the files are large so I don't... (7 Replies)
Discussion started by: rbatte1
7 Replies

4. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

5. Shell Programming and Scripting

Adding header to sub files after splitting the main file using AWK

Hi Folks, I have a file like: mainfile.txt: ------------- file1 abc def xyz file1 aaa pqr xyz file2 lmn ghi xyz file2 bbb tuv xyz I need output having two files file1 and file2. file1: ------ Name State Country abc def xyz aaa pqr xyz file2: (3 Replies)
Discussion started by: tanmay.gemini
3 Replies

6. Shell Programming and Scripting

Awk splitting words into files problem

Hi, I am trying to split the words having the delimiter as colon ';' in to separate files using awk. Here's my code. echo "f1;f2;f3" | awk '/;/{c=sprintf("%02d",++i); close("out" c)} {print > "out" c}' echo "f1;f2;f3" | awk -v i=0 '/;/{close("out"i); i++; next} {print > "out"i}' But... (4 Replies)
Discussion started by: royalibrahim
4 Replies

7. Shell Programming and Scripting

Apply 'awk' to all files in a directory or individual files from a command line

Hi All, I am using the awk command to replace ',' by '\t' (tabs) in a csv file. I would like to apply this to all .csv files in a directory and create .txt files with the tabs. How would I do this in a script? I have the following script called "csvtabs": awk 'BEGIN { FS... (4 Replies)
Discussion started by: ScKaSx
4 Replies

8. UNIX for Dummies Questions & Answers

Splitting files into a specific directory

Hello, I am trying to do the following; bzcat data.in.bz2 | split -l 1000000 -d this work great, except that once the files have been split, they are not in the directory I want them to be in. So I then have to move them, at times this can get hairy. Is there anyway to specify where the... (4 Replies)
Discussion started by: amcrisan
4 Replies

9. UNIX for Dummies Questions & Answers

Using Awk within awk to read all files in directory

I am wondering if anyone has any idea how to use an awk within awk to read files and find a match which adds to count. Say I am searching how many times the word crap appears in each files within a directory. How would i do that from the command prompt ... thanks (6 Replies)
Discussion started by: flevongo
6 Replies

10. Shell Programming and Scripting

Splitting input files into multiple files through AWK command

Hi, I needs to split *.txt files from single directory depends on the some mutltiple input values. i have wrote the code like below for file in *.txt do grep -i -h "value1|value2" $file > $file; done. My requirment is more input values needs to be given in grep; let us say 50... (3 Replies)
Discussion started by: arund_01
3 Replies
Login or Register to Ask a Question