I have over 900 files that have the same name except for a unique numeric assignment. For all files I would like to cut the 2nd column and paste all into one new file. All in bash.
sample input format for each file:
1 2 3
1 2 3
1 2 3
sample command for what I want to do:
cut -d' ' -f2 file_in >> file_out
Ultimately I would like process the files through a loop.
#!/bin/bash
for file in /home/epi/tmurray/reich_stuff/21
do cut -d' ' -f2 *.21 >> out.txt ; done
The above loop does the extraction but appends the column from each file vertically. How can I get it to append horizontally? e.g:
input:
file 1
1 2
1 2
1 2
file 2
1 2
1 2
1 2
current output
2
2
2
2
2
2
desired output:
2 2
2 2
2 2
thanks.
And Jim --that code you gave me doesn't work. I tweaked it and it still doesn't work. I keep getting errors about synthax
Your "loop" only loops over the directory, once, then ignores the directory when actually doing the cut, but never mind.
cut has a friend paste which places stuff next to each other. For a large number of files, it might not be workable, though; you'd need a temporary file, or a temporary stream, for each file in the set.
Sounds to me like at this point you would be best served by a simple awk or perl script.
I don't see anything wrong with the syntax in jim's command; can you post the actual error message?
For a small number of files, this works, at least roughly, but it causes my Bash to dump core with a malloc error when I try it on a large number of files. I'm posting it mainly for its curiosity value.
This constructs a command line consisting of paste <(cut file1) <(cut file2) <(cut file3) via some rather black magic. The key is really the use of eval and the quoting and backslashes required to pull it off.
Last edited by era; 05-12-2008 at 05:25 PM..
Reason: Belated typo fix
And here finally is a simple Perl one-liner (or to be really honest, two-liner) which collects an array for each input line number, and at the end prints out each array in line number order.
It's regrettable that you can't do this with just the basic Unix tools.
The close stuff is to reset the line numbering for each file (see perldoc -f eof) and the shift is to get rid of line number zero, which should be empty anyway. $F[1] is the second, space-padded input field, in case you want to change it to something else (Perl arrays are zero-based, so the first field is $F[0]). It's easy enough to make it split on a different separator if you like; see the -a and -F options. This uses references for efficiency (or simply because I'm past bedtime), so it's not particularly elegant or readable.
Given a directory containing say a few thousand files,
please output a list of all the names of the files in the directory that are exactly the same, i.e. have the same contents.
func(a_directory_name) output -> {“matches”: , ... ]}
e.g. func(“/home/my/files”) where the directory... (7 Replies)
Hi All,
I get ~1000 files everyday and the current unix script process one file at a time... like it unzips the file, loads into table . This process will repeat how many ever files were present in the directory. I was wondering can we process multiple files (like unzipping and loading X amount... (1 Reply)
Hi,
I am trying to print multiple patterns in a line using sed. But it is printing only the last occurance of a pattern.
If the line is
the the output should be
Lookup Procedure|Stored proc
But the output I am getting is
Stored proc
The code I am using is
echo... (9 Replies)
I try to trim a FTP log using awk, but the escape sequences does not work the way I want it.
Input data Wed 3Oct10 21:48:00 - (002117) Sent file d:\ftp\home\tools\htmake.rar successfully (48.2 kB/sec - 40997 Bytes)
Wed 3Oct10 22:25:46 - (002118) Sent file d:\ftp\files\main\oct\camera1... (4 Replies)
Hi
I always do a ftp in Unix to bring in a file from Mainframes. Then run a set of commands say 8 commands one by one to execute the brought in file. Upon completion of this commands, the data from the file gets transfered to Mainframe database. The problem is i can execute only 1 file at a time.... (2 Replies)
Hi I'm working on a file which looks like:
00001| |XXX|ABCD|27-SEP-1983|31-DEC-2099|Name|00001|N| |31|L| | | | |25-AUG-1990
I need to change the date format from DD-MMM-YYYY to MM/DD/YYYY
So i using
cat <filename> | sed... (2 Replies)
Hi all,
I have mistkanely gzipped twice an entire folder and sub folders, and also renamed the files during that process.
I am trying to undo this, and I need help to create the batch to work on it.
All folders are under my images directory, I have a output.txt file that holds all the... (1 Reply)
I have over 900 files that have the same name except for a unique numeric assignment. For all files I would like to cut the 2nd column and paste all into one new file. All in bash.
sample input format for each file:
1 2 3
1 2 3
1 2 3
sample command for what I want to do:
cut -d' ' -f2... (1 Reply)
Hi Gurus,
I am new to UNIX(HP). my requirmnet is File needs to needs to be split into multiple files dependa on one key value.
example
1 abc 333 us
2 bbc 444 ch
5 nnn 333 kk
7 hhh 555 ll
3 hhh 333 hh
now the requirment is line with 333 should be copied into test1.txt and... (14 Replies)