Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Code to pick all files from a directory and send it to a command

Login or Register to Reply

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Code to pick all files from a directory and send it to a command
# 1  
Old 12-15-2013
Code to pick all files from a directory and send it to a command
# 2  
Old 12-15-2013
Try something like this:

Code:
find . -maxdepth 1 -name "*.txt" -type f -print0 |
   sed -E 's/(^|\x0)([^$])/\1--GATKdepths\x0\2/g' |
   xargs -r -0 ./command --mergeGATKdepths -o ./DATA.RD.txt

However, Be careful as there are limits to the command line length.

4096 characters is a typical limit and this would allow around:
90 files with an average filename length of 30 characters or
160 files with names 10 chars in length.
Last edited by Chubler_XL; 12-16-2013 at 12:28 AM.. Reason: Avoid extra --GATKdepths after last file
# 3  
Old 12-16-2013
Thanks for the reply. But what if I have more than 100 files and the length of name is suppose 50 characters(includes the path).

---------- Post updated at 04:25 PM ---------- Previous update was at 03:35 PM ----------

I added 1 more command to replace new line with space
Code:
find . -maxdepth 1 -name "*.txt" -type f -print0 | sed -E 's/(^|\x0)([^$])/\1--GATKdepths\x0\2/g' |sed ':a;N;$!ba;s/\n/ /g' | xargs -r -0 ./xhmm --mergeGATKdepths -o ./DATA.RD.txt

But I get the following error :
Code:
xhmm: unrecognized option '--GATKdepths '

What is the reason for it?
Last edited by Franklin52; 12-17-2013 at 03:50 AM.. Reason: Please use code tags
# 4  
Old 12-16-2013
Here is another approach in bash:
Code:
#!/bin/bash

for file in *.txt
do
        if [ -z "$param" ]
        then
                param=$( printf "%sGATKdepths %s" "--" "$file" )
        else
                param=$( printf "%s \\ \n%sGATKdepths %s" "$param" "--" "$file" )
        fi
done

./xhmm --mergeGATKdepths -o ./DATA.RD.txt "$param"

# 5  
Old 12-16-2013
Quote:
Originally Posted by rossi
Thanks for the reply. But what if I have more than 100 files and the length of name is suppose 50 characters(includes the path).
Depending on your OS you may on may not get away with 1 command to do this. can you split the calls to xhmm up with say 25 file each?

Quote:
Originally Posted by rossi
I added 1 more command to replace new line with space

find . -maxdepth 1 -name "*.txt" -type f -print0 | sed -E 's/(^|\x0)([^$])/\1--GATKdepths\x0\2/g' |sed ':a;N;$!ba;s/\n/ /g' | xargs -r -0 ./xhmm --mergeGATKdepths -o ./DATA.RD.txt

But I get the following error :
xhmm: unrecognized option '--GATKdepths '
What is the reason for it?
Looks like you sed is having trouble replacing null char. Do you have files with spaces in their names? If not this might work out:

Code:
./xhmm --mergeGATKdepths -0 ./DATA.RD.txt $(printf -- "--GATKdepths %s " *.txt)

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Pick up files to process using ls command

Hello All, I have a list of files as below --------- FORM_PTIR_9484_ANFDOP_20150716_00263059.DAT FORM_PTIR_9484_TIFDOP_20150716_00263059.DAT FORM_PTIR_9484_EXFDOP_20150716_00263059.DAT FORM_PTIR_9484_ANFDEX_20150716_00263059.DAT FORM_PTIR_9484_ANFDCA_20150716_00263059.DAT and so on....... (4 Replies)
Discussion started by: rac
4 Replies

2. UNIX for Dummies Questions & Answers

Send output of command to screen and to multiple files

Hi there, I'm a newbie to Unix (taking a course in it right now) and I don't know how to do the following in bash: I need to write a command to display information about the used and free space on the file system, showing only local file systems, and then send the output of the command to... (1 Reply)
Discussion started by: damianberry
1 Replies

3. Shell Programming and Scripting

to pick the latest file modified in a directory

I wan to pick the latest modified file name and redirect it to a file .. ls -tr | tail -1 >file but this is printing file ins side the filename , can anyone help me out (5 Replies)
Discussion started by: vishwakar
5 Replies

4. Shell Programming and Scripting

Apply 'awk' to all files in a directory or individual files from a command line

Hi All, I am using the awk command to replace ',' by '\t' (tabs) in a csv file. I would like to apply this to all .csv files in a directory and create .txt files with the tabs. How would I do this in a script? I have the following script called "csvtabs": awk 'BEGIN { FS... (4 Replies)
Discussion started by: ScKaSx
4 Replies

5. Shell Programming and Scripting

Find the latest directory and loop through the files and pick the error messages

Hi, I am new to unix and shell scripting,can anybody help me in sctipting a requirement. my requirement is to get the latest directory the name of the directory will be like CSB.monthdate_time stamp like CSB.Sep29_11:16 and CSB.Oct01_16:21. i need to pick the latest directory. in the... (15 Replies)
Discussion started by: sudhir_83k
15 Replies

6. Shell Programming and Scripting

Pick random file from ls command.

Lets say I want to pick a random file when I do an "ls" command. I don't have set number of files in each directory. ls | head -1 This gives me the first one in each directory, is there a way to do the same but pick a random one. (3 Replies)
Discussion started by: elbombillo
3 Replies

7. Shell Programming and Scripting

Use awk to pick out zip code

Hi, Suppose I have a csv file, each line look like this: ABC Company, 1999, March, caucasian owned, 123 BroadWay NY 92939-2222 How do I create two new columns at the end, one for state, one for zip. So that the line is ABC Company, 1999, March, caucasian owned, 123 BroadWay NY... (2 Replies)
Discussion started by: grossgermany
2 Replies

8. UNIX for Dummies Questions & Answers

pick the very latest directory

Hi, I have some list of directories in the form datemonthyear e.g. 02082009, 03082009 and 04082009 etc. I need to pick the latest directory from the current working directory. Outcome: 05082009 This is the output am expecting. Thanks (6 Replies)
Discussion started by: venkatesht
6 Replies

9. UNIX for Dummies Questions & Answers

to pick up the Return Code ( RC) from the mailx command and return it to SAS uisng 's

Hi All, Can anyone please let me know the syntax / how to pick up the Return Code ( RC) from the mailx command and return it to SAS uisng 'system()' function and '${?}'. I am in a process to send the mail automatically with an attachment to bulk users. I have used 'Mailx' and 'Unencode'... (0 Replies)
Discussion started by: manas6
0 Replies
Login or Register to Ask a Question