Sponsored Content
Top Forums UNIX for Dummies Questions & Answers How do I Process Multiple Files Post 302135901 by dc1743 on Thursday 13th of September 2007 08:32:34 AM
Old 09-13-2007
How do I Process Multiple Files

Hi,

First post here.

In Mac OSX terminal I need to run a program against multiple files in a directory and append the output to tab separated variable file.

I currently type the following to process just one file

Code:
MBP:/Users/dc1743/Desktop/SFT root# ./myprogram myfile1.plist >> output.tsv

My program and all the files I want to process are in the SFT directory in this example. I want to be able to carry out this process against all of my files in one go rather than one at a time. Please can anyone suggest a command or simple script to achieve this.

As I am sure you can tell I am a complete novice at this.

Regards Richard
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to process multiple files in Korn Shell

How do I make the below ksh to process all of the files inside a user specified directory? Currently it can only process one file at a time. #!/bin/ksh tr -s '\11 ' ' ' < $1 > temp0 sed -e 's/,//g' temp0 > temp1 cut -d' ' -f1,4,5 temp1 > final_output rm temp0 temp1 (3 Replies)
Discussion started by: stevefox
3 Replies

2. IP Networking

How to choose Multiple process or Multiple threads?

Hi All, Please explain me when i have to use multiple process and when I have to use Multiple threads? Please give me an example.It will be very helpful for me. Thanks in advance. (0 Replies)
Discussion started by: ashleykumar
0 Replies

3. Programming

Process multiple pcap files at once

Hi all, I'm writing a program using libpcap, and I have multiple pcap files in a folder that I want to capture. I currently have handle = pcap_open_offline("/data/traffic/pcap1.pcap", errbuf"); which works fine since pcap_open_offline() takes in a filename. However, I want to process... (0 Replies)
Discussion started by: lancer6238
0 Replies

4. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is... (3 Replies)
Discussion started by: Liverpaul09
3 Replies

5. Shell Programming and Scripting

How to process multiple input files using Shell scripting

Hi, I would like to write a for loop that does the following: I have a file called X.txt and other files called 1.txt,2.txt, .....,1000.txt. I want to substitute the 6th column of the file X.txt with 1.txt and store the output as X.1. Then I want to do the same with X.txt and 2.txt and store... (0 Replies)
Discussion started by: evelibertine
0 Replies

6. UNIX for Dummies Questions & Answers

Writing a loop to process multiple input files by a shell script

I have multiple input files that I want to manipulate using a shell script. The files are called 250.1 through 250.1000 but I only want the script to manipulate 250.300 through 250.1000. Before I was using the following script to manipulate the text files: for i in 250.*; do || awk... (4 Replies)
Discussion started by: evelibertine
4 Replies

7. Shell Programming and Scripting

awk, multiple files input and multiple files output

Hi! I'm new in awk and I need some help. I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files. Thanks in advance for help! :-) ciao (5 Replies)
Discussion started by: gabrysfe
5 Replies

8. UNIX for Advanced & Expert Users

How to process multiple files parallely

Hi, I have a requirement to process multiple files in a directory parallely.Consider the below scenario: In a directory there are three files file1,file2 and file3.When I use for loop each file will be executed in sequence but I want to process parallely. Any Help would be appreciated.... (1 Reply)
Discussion started by: liyakathali
1 Replies

9. Shell Programming and Scripting

Process multiple large files with awk

Hi there, I'm camor and I'm trying to process huge files with bash scripting and awk. I've got a dataset folder with 10 files (16 millions of row each one - 600MB), and I've got a sorted file with all keys inside. For example: a sample_1 200 a.b sample_2 10 a sample_3 10 a sample_1 10 a... (4 Replies)
Discussion started by: camor
4 Replies

10. Shell Programming and Scripting

Xargs to call python executable to process multiple bam files

I am running the below loop that to process the 3 bam files (which isn't always the case). A .py executable is then called using | xargs sh to further process. If I just run it with echo the output is fine and expected, however when | xargs sh is added I get the error. I tried adding | xargs... (4 Replies)
Discussion started by: cmccabe
4 Replies
HWLOC-PS(1)							       hwloc							       HWLOC-PS(1)

NAME
hwloc-ps - List currently-running processes or threads that are bound SYNOPSIS
hwloc-ps [options] OPTIONS
-a list all processes, even those that are not bound to any specific part of the machine. -p --physical report OS/physical indexes instead of logical indexes -l --logical report logical indexes instead of physical/OS indexes (default) -c --cpuset show process bindings as cpusets instead of objects. -t --threads show threads inside processes. If -a is given as well, list all threads within each process. Otherwise, show all threads inside each process where at least one thread is bound. --whole-system Do not consider administration limitations. --pid-cmd <cmd> Append the output of the given command to each PID line. For each displayed process ID, execute the command <cmd> <pid> and append the first line of its output to the regular hwloc-ps line. DESCRIPTION
By default, hwloc-ps lists only those currently-running processes that are bound. If -t is given, processes that are not bound but contain at least one bound thread are also displayed, as well as all their threads. hwloc-ps displays process identifier, command-line and binding. The binding may be reported as objects or cpusets. By default, process bindings are restricted to the currently available topology. If some processes are bound to processors that are not available to the current process, they are ignored unless --whole-system is given. The output is a plain list. If you wish to annotate the hierarchical topology with processes so as to see how they are actual distributed on the machine, you might want to use lstopo --ps instead (which also only shows processes that are bound). The -a switch can be used to show all processes, if desired. EXAMPLES
If a process is bound, it appears in the default output: $ utils/hwloc-ps 4759 Core:0 myprogram If a process is not bound but 3 of his 4 threads are bound, it only appears in the thread-aware output: $ utils/hwloc-ps $ utils/hwloc-ps -t 4759 Machine:0 myprogram 4759 Machine:0 4761 PU:0 4762 PU:2 4765 PU:1 To display the binding of already running MPI processes (launched by Open MPI) and append their MPI rank (in MPI_COMM_WORLD) to each line: $ utils/hwloc-ps --pid-cmd myscript 29093 L1dCache:0 myprogram OMPI_COMM_WORLD_RANK=0 29094 L1dCache:2 myprogram OMPI_COMM_WORLD_RANK=1 29095 L1dCache:1 myprogram OMPI_COMM_WORLD_RANK=2 29096 L1dCache:3 myprogram OMPI_COMM_WORLD_RANK=3 where myscript is a bash script doing: #!/bin/sh cat /proc/$1/environ 2>/dev/null | xargs --null --max-args=1 echo | grep OMPI_COMM_WORLD_RANK SEE ALSO
hwloc(7), lstopo(1), hwloc-calc(1), hwloc-distrib(1) 1.7 Apr 07, 2013 HWLOC-PS(1)
All times are GMT -4. The time now is 08:49 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy