10-12-2010
Using AWK: Extract data from multiple files and output to multiple new files
Hi,
I'd like to process multiple files. For example:
file1.txt
file2.txt
file3.txt
Each file contains several lines of data. I want to extract a piece of data and output it to a new file.
file1.txt ----> newfile1.txt
file2.txt ----> newfile2.txt
file3.txt ----> newfile3.txt
Here is an example of what an input file and an output file could look like:
file1.txt
---------
line1
line2
line3
newfile1.txt
------------
line2
So, in this case, the awk script would extract line2 from each input file. That part is ok. I need to know how to create the multiple output files which contain the extracted data (ie. line2).
Any help would be greatly appreciated
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hello,
I am trying to write a bash shell script that does the following:
1.Finds all *.txt files within my directory of interest
2. reads each of the files (25 files) one by one (tab-delimited format and have the same data format)
3. skips the first 10 rows of the file
4. extracts and... (4 Replies)
Discussion started by: manishabh
4 Replies
2. Shell Programming and Scripting
hello,
I will would be grateful if anyone can help me reply to my post
extract multiple cloumns from multiple files; skip rows and include filenames; awk
Please see this thread.
Thanks
manishabh (0 Replies)
Discussion started by: manishabh
0 Replies
3. UNIX for Dummies Questions & Answers
Hi,
I'm using AWK to try to extract data from multiple files (*.txt). The script should look for a flag that occurs at a specific position in each file and it should return the data to the right of that flag.
I should end up with one line for each file, each containing 3 columns:... (8 Replies)
Discussion started by: Liverpaul09
8 Replies
4. UNIX for Dummies Questions & Answers
Hi,
I have a directory full of *.txt files. I would like to print the last line of every file to screen.
I know you can use FNR for printing the first line of each file, but how do I access the last line of each file?
This code doesn't work, it only prints the last line of the last file:BEGIN... (5 Replies)
Discussion started by: Liverpaul09
5 Replies
5. UNIX for Dummies Questions & Answers
Hi All. Thanks for your help in advance.
I have a requirement to examine the number of delimiters in each record of a file. If the record has the expected number of delimiters it should be passed into a 'good' file. If it does not, the record should be passed into a 'bad' file. I have been able... (8 Replies)
Discussion started by: codestar1
8 Replies
6. Shell Programming and Scripting
Hi!
I'm new in awk and I need some help.
I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files.
Thanks in advance for help!
:-)
ciao (5 Replies)
Discussion started by: gabrysfe
5 Replies
7. UNIX for Dummies Questions & Answers
I am trying to extract common list of Organisms from different files
For example I took 3 files and showed expected result. In real I have more than 1000 files. I am aware about the useful use of awk and grep but unaware in depth so need guidance regarding it.
I want to use awk/ grep/ cut/... (7 Replies)
Discussion started by: macmath
7 Replies
8. Shell Programming and Scripting
Hi,
I have directory with multiple files from which i need to extract portion of specif lines and insert it in a new file, the new file will contain a separate columns for each file data.
Example:
I need to extract Value_1 & Value_3 from all files and insert in output file as below:
... (2 Replies)
Discussion started by: belalr
2 Replies
9. Shell Programming and Scripting
Dear All,
I am trying to write a Unix Script which fires a sql query. The output of the sql query gives multiple rows. Each row should be saved in a separate Unix File.
The number of rows of sql output can be variable. I am able save all the rows in one file but in separate files.
Any... (14 Replies)
Discussion started by: Rahul_Bhasin
14 Replies
10. Shell Programming and Scripting
Hi All,
I want to use egrep on multiple files and the results should be output to multiple files. I am using the below code in my shell script(working in Ksh shell). However with this code I am not attaining the desired results.
#!/bin/ksh
(
a="/path/file1"
b="path/file2"
for file in... (4 Replies)
Discussion started by: am24
4 Replies
LEARN ABOUT REDHAT
dos2unix
dos2unix(1) General Commands Manual dos2unix(1)
NAME
dos2unix - DOS/MAC to UNIX text file format converter
SYNOPSYS
dos2unix [options] [-c convmode] [-o file ...] [-n infile outfile ...]
Options:
[-hkqV] [--help] [--keepdate] [--quiet] [--version]
DESCRIPTION
This manual page documents dos2unix, the program that converts plain text files in DOS/MAC format to UNIX format.
OPTIONS
The following options are available:
-h --help
Print online help.
-k --keepdate
Keep the date stamp of output file same as input file.
-q --quiet
Quiet mode. Suppress all warning and messages.
-V --version
Prints version information.
-c --convmode convmode
Sets conversion mode. Simulates dos2unix under SunOS.
-o --oldfile file ...
Old file mode. Convert the file and write output to it. The program default to run in this mode. Wildcard names may be used.
-n --newfile infile outfile ...
New file mode. Convert the infile and write output to outfile. File names must be given in pairs and wildcard names should NOT be
used or you WILL lost your files.
EXAMPLES
Get input from stdin and write output to stdout.
dos2unix
Convert and replace a.txt. Convert and replace b.txt.
dos2unix a.txt b.txt
dos2unix -o a.txt b.txt
Convert and replace a.txt in ASCII conversion mode. Convert and replace b.txt in ISO conversion mode. Convert c.txt from Mac to Unix
ascii format.
dos2unix a.txt -c iso b.txt
dos2unix -c ascii a.txt -c iso b.txt
dos2unix -c mac a.txt b.txt
Convert and replace a.txt while keeping original date stamp.
dos2unix -k a.txt
dos2unix -k -o a.txt
Convert a.txt and write to e.txt.
dos2unix -n a.txt e.txt
Convert a.txt and write to e.txt, keep date stamp of e.txt same as a.txt.
dos2unix -k -n a.txt e.txt
Convert and replace a.txt. Convert b.txt and write to e.txt.
dos2unix a.txt -n b.txt e.txt
dos2unix -o a.txt -n b.txt e.txt
Convert c.txt and write to e.txt. Convert and replace a.txt. Convert and replace b.txt. Convert d.txt and write to f.txt.
dos2unix -n c.txt e.txt -o a.txt b.txt -n d.txt f.txt
DIAGNOSTICS
BUGS
The program does not work properly under MSDOS in stdio processing mode. If you know why is that so, please tell me.
AUTHORS
Benjamin Lin - <blin@socs.uts.edu.au>
Bernd Johannes Wuebben (mac2unix mode) <wuebben@kde.org>
MISCELLANY
Tested environment:
Linux 1.2.0 with GNU C 2.5.8
SunOS 4.1.3 with GNU C 2.6.3
MS-DOS 6.20 with Borland C++ 4.02
Suggestions and bug reports are welcome.
SEE ALSO
unix2dos(1) mac2unix(1)
1995.03.31 dos2unix v3.0 dos2unix(1)