Sponsored Content
Full Discussion: Looping through input/output
Top Forums UNIX for Beginners Questions & Answers Looping through input/output Post 302979882 by zajtat on Saturday 20th of August 2016 08:30:52 PM
Old 08-20-2016
I'm sorry my explanations were not clear.

The executable command takes the columns of file specified in argument 1 (infile1) and adds them to the file specified in argument 2 (infile2). It creates a new file that starts with columns from infile2, followed by columns from infile1. So, the first command line is:

combining.executable infile1 infile2 > outfile1

This will take infile2 as the basis and add columns to it from infile1. The outfile1 will start with columns from infile2 and finish with columns from infile1.

The we run the following command:

combining.executable infile3 outfile1 > outfile2

This will take the columns from the file specified as argument 1 (infile3) and add them to the file specified in the argument 2 (outfile1). It is the same procedure as in step1, so the program does the same thing, nothing changes. The resulting outfile2 will start with columns from outfile1 and finish with columns from infile3. So, all 3 files are combined and nothing is deleted/replaced.

For the next step, we'll need outfile2 and infile4. So, we can delete the outfile1 as its info is now in outfile2. And so on...

This is your next question:

What is the format of the real names of your input files? (Please show us the actual names of your 1st input file and your last input file.)

The answer: the files are simple text files, here is the name of my first input file:

9464294024_R01C01header

and the last file name is:

9479475073_R12C02header

here is a small subsample of the files:

Code:
9464294024_R01C01header  9477371149_R12C02header  9477871078_R06C01header  9477875165_R03C01header  9477885102_R05C01header  9479475073_R10C02header
9464294024_R01C02header  9477371157_R01C01header  9477871078_R06C02header  9477875165_R03C02header  9477885102_R05C02header  9479475073_R11C01header

Next question: And, is the number of input files a constant?

I'm not sure I understand this question. There is exactly 5427 files than need to be combined into one.

Next question: And, what name do you really want for your output file?

The name of the output file does not really matter for me. It can be anything as long as it contains the columns of all the 5427 files combined into one text file.

Many thanks!

Last edited by Don Cragun; 08-20-2016 at 09:53 PM.. Reason: Change ICODE tags to CODE tags for long data lines.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Using Output from one command as input to another

This site has been very helpful thus far.. I thank you all in advance for sharing the knowledge. Let me get to it. I am trying to write a very small script to take away from the boredom of doing the same thing over and over. Everynow and again I have to get the hex value of a file using a... (2 Replies)
Discussion started by: BkontheShell718
2 Replies

2. UNIX for Advanced & Expert Users

input/Output settings

How can we view all the input/output settings of unix environment for a session (6 Replies)
Discussion started by: paritoshc
6 Replies

3. Shell Programming and Scripting

input -output file

Hi, I am having an Input file .which is having a list of names. comapring with our database , needs to write the out put in file called output.txt , format should be name--> country--->phone number could you please help me.. thanks in advance (7 Replies)
Discussion started by: hegdeshashi
7 Replies

4. Shell Programming and Scripting

perl: looping through the output of a 'system' command

Hi there could anybody point me in the right direction when it comes to looping through the output of a system command in perl (i.e. df -k) doing a test against each line to see if it matches? for example if i have a df -k output like this and I wanted to grab the lines that matched "sda" or... (3 Replies)
Discussion started by: rethink
3 Replies

5. Shell Programming and Scripting

Dynamic output file generation using a input text file with predefined output format

Hi, I have two files , one file with data file with attributes that need to be sent to another file to generate a predefined format. Example: File.txt AP|{SSHA}VEEg42CNCghUnGhCVg== APVG3|{SSHA}XK|"password" AP3|{SSHA}XK|"This is test" .... etc --------- test.sh has... (1 Reply)
Discussion started by: hudson03051nh
1 Replies

6. Solaris

SVM Solaris 8 Problem. Metastat output looping

Hi friends, I'm newbie to SVM. Just wanna try installed it on one of our server (to do mirroring for disk0 and disk1) but i think im lost until now. :( the steps i've taken is as below:- 1.prtvtoc /dev/rdsk/c1t0d0s2 | fmthard -s - /dev/rdsk/c1t1d0s2 2.metadb -a -c 3 -f c1t0d0s7... (3 Replies)
Discussion started by: kronenose
3 Replies

7. Shell Programming and Scripting

Looping through for user input

Legends, I want to remain in the script until user passes the correct name. I had tried the below code; but it didn't work out. Please help echo "\nPlease enter the source system: \c" while read SYSTEM_NAME do if ]; then echo "\nMaking $SYSTEM_NAME as source system for particular... (5 Replies)
Discussion started by: sdosanjh
5 Replies

8. Shell Programming and Scripting

looping and saving output of each line separately

I have been trying this program for a long time. I am trying to read a file named "odon" line by line; read the first line, send it to do a command saved in a file "perm", once the first line has finished going through the content of the file perm, the result is saved with the number of the line.... (17 Replies)
Discussion started by: iconig
17 Replies

9. UNIX for Dummies Questions & Answers

Looping through the contents of array for output name

Hi all, I am trying to loop through the string contents of an array, to add it during the saving of the output files. I am trying this code to print each column and save it to unique file name, but it doesn't work. Thanks for any help. fnam=(japan usa uk) alldata.dat contained sample data... (1 Reply)
Discussion started by: ida1215
1 Replies

10. Shell Programming and Scripting

Looping over output of 'ls'

Hi, I have some output from 'ls' command and I want to loop over the output in a bash script. What would be a good way to go about it? For example, if the output of the ls command gives me 'prefix1 prefix2 prefix3', how can I set a loop that will iterate over these? many thanks! (5 Replies)
Discussion started by: pc2001
5 Replies
sort(1) 							   User Commands							   sort(1)

NAME
sort - sort, merge, or sequence check text files SYNOPSIS
/usr/bin/sort [-bcdfimMnru] [-k keydef] [-o output] [-S kmem] [-t char] [-T directory] [ -y [kmem]] [-z recsz] [ +pos1 [-pos2]] [file...] /usr/xpg4/bin/sort [-bcdfimMnru] [-k keydef] [-o output] [-S kmem] [-t char] [-T directory] [ -y [kmem]] [-z recsz] [ +pos1 [-pos2]] [file...] DESCRIPTION
The sort command sorts lines of all the named files together and writes the result on the standard output. Comparisons are based on one or more sort keys extracted from each line of input. By default, there is one sort key, the entire input line. Lines are ordered according to the collating sequence of the current locale. OPTIONS
The following options alter the default behavior: /usr/bin/sort -c Checks that the single input file is ordered as specified by the arguments and the collating sequence of the current locale. The exit code is set and no output is produced unless the file is out of sort. /usr/xpg4/bin/sort -c Same as /usr/bin/sort except no output is produced under any circumstances. -m Merges only. The input files are assumed to be already sorted. -o output Specifies the name of an output file to be used instead of the standard output. This file can be the same as one of the input files. -S kmem Specifies the maximum amount of swap-based memory used for sorting, in kilobytes (the default unit). kmem can also be spec- ified directly as a number of bytes (b), kilobytes (k), megabytes (m), gigabytes (g), or terabytes (t); or as a percentage (%) of the installed physical memory. -T directory Specifies the directory in which to place temporary files. -u Unique: suppresses all but one in each set of lines having equal keys. If used with the -c option, checks that there are no lines with duplicate keys in addition to checking that the input file is sorted. -y kmem (obsolete). This option was used to specify the amount of main memory initially used by sort. Its functionality is not appropriate for a virtual memory system; memory usage for sort is now specified using the -S option. -z recsz (obsolete). This option was used to prevent abnormal termination when lines longer than the system-dependent default buffer size are encountered. Because sort automatically allocates buffers large enough to hold the longest line, this option has no effect. Ordering Options The default sort order depends on the value of LC_COLLATE. If LC_COLLATE is set to C, sorting will be in ASCII order. If LC_COLLATE is set to en_US, sorting is case insensitive except when the two strings are otherwise equal and one has an uppercase letter earlier than the other. Other locales will have other sort orders. The following options override the default ordering rules. When ordering options appear independent of any key field specifications, the requested field ordering rules are applied globally to all sort keys. When attached to a specific key (see Sort Key Options), the specified ordering options override all global ordering options for that key. In the obsolescent forms, if one or more of these options follows a +pos1 option, it will affect only the key field specified by that preceding option. -d ``Dictionary'' order: only letters, digits, and blanks (spaces and tabs) are significant in comparisons. -f Folds lower-case letters into upper case. -i Ignores non-printable characters. -M Compares as months. The first three non-blank characters of the field are folded to upper case and compared. For example, in English the sorting order is "JAN" < "FEB" < ... < "DEC". Invalid fields compare low to "JAN". The -M option implies the -b option (see below). -n Restricts the sort key to an initial numeric string, consisting of optional blank characters, optional minus sign, and zero or more digits with an optional radix character and thousands separators (as defined in the current locale), which will be sorted by arithmetic value. An empty digit string is treated as zero. Leading zeros and signs on zeros do not affect ordering. -r Reverses the sense of comparisons. Field Separator Options The treatment of field separators can be altered using the following options: -b Ignores leading blank characters when determining the starting and ending positions of a restricted sort key. If the -b option is specified before the first sort key option, it is applied to all sort key options. Otherwise, the -b option can be attached independently to each -k field_start, field_end, or +pos1 or -pos2 option-argument (see below). -t char Use char as the field separator character. char is not considered to be part of a field (although it can be included in a sort key). Each occurrence of char is significant (for example, <char><char> delimits an empty field). If -t is not speci- fied, blank characters are used as default field separators; each maximal non-empty sequence of blank characters that fol- lows a non-blank character is a field separator. Sort Key Options Sort keys can be specified using the options: -k keydef The keydef argument is a restricted sort key field definition. The format of this definition is: -k field_start [type] [,field_end [type] ] where: field_start and field_end define a key field restricted to a portion of the line. type is a modifier from the list of characters bdfiMnr. The b modifier behaves like the -b option, but applies only to the field_start or field_end to which it is attached and characters within a field are counted from the first non-blank character in the field. (This applies separately to first_character and last_character.) The other modifiers behave like the corresponding options, but apply only to the key field to which they are attached. They have this effect if specified with field_start, field_end or both. If any modifier is attached to a field_start or to a field_end, no option applies to either. When there are multiple key fields, later keys are compared only after all earlier keys compare equal. Except when the -u option is specified, lines that otherwise compare equal are ordered as if none of the options -d, -f, -i, -n or -k were present (but with -r still in effect, if it was specified) and with all bytes in the lines significant to the comparison. The notation: -k field_start[type][,field_end[type]] defines a key field that begins at field_start and ends at field_end inclusive, unless field_start falls beyond the end of the line or after field_end, in which case the key field is empty. A missing field_end means the last character of the line. A field comprises a maximal sequence of non-separating characters and, in the absence of option -t, any preceding field separator. The field_start portion of the keydef option-argument has the form: field_number[.first_character] Fields and characters within fields are numbered starting with 1. field_number and first_character, interpreted as positive decimal integers, specify the first character to be used as part of a sort key. If .first_character is omitted, it refers to the first character of the field. The field_end portion of the keydef option-argument has the form: field_number[.last_character] The field_number is as described above for field_start. last_character, interpreted as a non-negative decimal integer, specifies the last character to be used as part of the sort key. If last_character evaluates to zero or .last_character is omitted, it refers to the last character of the field specified by field_number. If the -b option or b type modifier is in effect, characters within a field are counted from the first non-blank character in the field. (This applies separately to first_character and last_character.) [+pos1 [-pos2]] (obsolete). Provide functionality equivalent to the -kkeydef option. pos1 and pos2 each have the form m.n optionally followed by one or more of the flags bdfiMnr. A starting position specified by +m.n is interpreted to mean the n+1st character in the m+1st field. A missing .n means .0, indicating the first charac- ter of the m+1st field. If the b flag is in effect n is counted from the first non-blank in the m+1st field; +m.0b refers to the first non-blank character in the m+1st field. A last position specified by -m.n is interpreted to mean the nth character (including separators) after the last character of the mth field. A missing .n means .0, indicating the last character of the mth field. If the b flag is in effect n is counted from the last leading blank in the m+1st field; -m.1b refers to the first non-blank in the m+1st field. The fully specified +pos1 -pos2 form with type modifiers T and U: +w.xT -y.zU is equivalent to: undefined (z==0 & U contains b & -t is present) -k w+1.x+1T,y.0U (z==0 otherwise) -k w+1.x+1T,y+1.zU (z > 0) Implementations support at least nine occurrences of the sort keys (the -k option and obsolescent +pos1 and -pos2) which are significant in command line order. If no sort key is specified, a default sort key of the entire line is used. OPERANDS
The following operand is supported: file A path name of a file to be sorted, merged or checked. If no file operands are specified, or if a file operand is -, the standard input will be used. USAGE
See largefile(5) for the description of the behavior of sort when encountering files greater than or equal to 2 Gbyte ( 2**31 bytes). EXAMPLES
In the following examples, first the preferred and then the obsolete way of specifying sort keys are given as an aid to understanding the relationship between the two forms. Example 1: Sorting with the second field as a sort key Either of the following commands sorts the contents of infile with the second field as the sort key: example% sort -k 2,2 infile example% sort +1 -2 infile Example 2: Sorting in reverse order Either of the following commands sorts, in reverse order, the contents of infile1 and infile2, placing the output in outfile and using the second character of the second field as the sort key (assuming that the first character of the second field is the field separator): example% sort -r -o outfile -k 2.2,2.2 infile1 infile2 example% sort -r -o outfile +1.1 -1.2 infile1 infile2 Example 3: Sorting using a specified character in one of the files Either of the following commands sorts the contents of infile1 and infile2 using the second non-blank character of the second field as the sort key: example% sort -k 2.2b,2.2b infile1 infile2 example% sort +1.1b -1.2b infile1 infile2 Example 4: Sorting by numeric user ID Either of the following commands prints the passwd(4) file (user database) sorted by the numeric user ID (the third colon-separated field): example% sort -t : -k 3,3n /etc/passwd example% sort -t : +2 -3n /etc/passwd Example 5: Printing sorted lines excluding lines that duplicate a field Either of the following commands prints the lines of the already sorted file infile, suppressing all but one occurrence of lines having the same third field: example% sort -um -k 3.1,3.0 infile example% sort -um +2.0 -3.0 infile Example 6: Sorting by host IP address Either of the following commands prints the hosts(4) file (IPv4 hosts database), sorted by the numeric IP address (the first four numeric fields): example$ sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n /etc/hosts example$ sort -t . +0 -1n +1 -2n +2 -3n +3 -4n /etc/hosts Since '.' is both the field delimiter and, in many locales, the decimal separator, failure to specify both ends of the field will lead to results where the second field is interpreted as a fractional portion of the first, and so forth. ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of sort: LANG, LC_ALL, LC_COLLATE, LC_MES- SAGES, and NLSPATH. LC_CTYPE Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single- versus multi-byte characters in arguments and input files) and the behavior of character classification for the -b, -d, -f, -i and -n options. LC_NUMERIC Determine the locale for the definition of the radix character and thousands separator for the -n option. EXIT STATUS
The following exit values are returned: 0 All input files were output successfully, or -c was specified and the input file was correctly sorted. 1 Under the -c option, the file was not ordered as specified, or if the -c and -u options were both specified, two input lines were found with equal keys. >1 An error occurred. FILES
/var/tmp/stm??? temporary files ATTRIBUTES
See attributes(5) for descriptions of the following attributes: /usr/bin/sort +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWesu | +-----------------------------+-----------------------------+ |CSI |Enabled | +-----------------------------+-----------------------------+ /usr/xpg4/bin/sort +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWxcu4 | +-----------------------------+-----------------------------+ |CSI |Enabled | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ SEE ALSO
comm(1), join(1), uniq(1), nl_langinfo(3C), strftime(3C), hosts(4), passwd(4), attributes(5), environ(5), largefile(5), standards(5) DIAGNOSTICS
Comments and exits with non-zero status for various trouble conditions (for example, when input lines are too long), and for disorders dis- covered under the -c option. NOTES
When the last line of an input file is missing a new-line character, sort appends one, prints a warning message, and continues. sort does not guarantee preservation of relative line ordering on equal keys. One can tune sort performance for a specific scenario using the -S option. However, one should note in particular that sort has greater knowledge of how to use a finite amount of memory for sorting than the virtual memory system. Thus, a sort invoked to request an extremely large amount of memory via the -S option could perform extremely poorly. As noted, certain of the field modifiers (such as -M and -d) cause the interpretation of input data to be done with reference to locale- specific settings. The results of this interpretation can be unexpected if one's expectations are not aligned with the conventions estab- lished by the locale. In the case of the month keys, sort does not attempt to compensate for "approximate" month abbreviations. The precise month abbreviations from nl_langinfo(3C) or strftime(3C) are the only ones recognized. For printable or dictionary order, if these concepts are not well-defined by the locale, an empty sort key may be the result, leading to the next key being the significant one for determining the appropriate ordering. SunOS 5.10 19 Nov 2001 sort(1)
All times are GMT -4. The time now is 02:03 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy