Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Combine multiple files with common string into one new file. Post 302465118 by GradStudent2010 on Thursday 21st of October 2010 04:41:08 PM
Old 10-21-2010
Combine multiple files with common string into one new file.

I need to compile a large amount of data with a common string from individual text files throughout many directories.


An example data file is below. I want to search for the following string, "cc_sectors_1" and combine all the data from each file which contains this string, into one new output file.
Code:
cc_sectors_1_pt2.nii.gz
34 408.000000 0.582149 0.165659 0.250764 0.783992 
cc_sectors_2_pt2.nii.gz
10 120.000000 0.515655 0.140343 0.329384 0.711448 
cc_sectors_3_pt2.nii.gz
11 132.000000 0.552913 0.178691 0.266971 0.775907 
cc_sectors_4_pt2.nii.gz
15 180.000000 0.522139 0.174769 0.255384 0.728975 
cc_sectors_5_pt2.nii.gz
41 492.000000 0.511493 0.173014 0.220017 0.720323

I am assuming this will be an awk command. Any suggestions?

I appreciate any help!

Last edited by Scott; 10-25-2010 at 03:24 AM.. Reason: Please use code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Combine multiple columns from multiple files

Hi there, I was wondering if someone can help me with this. I am trying the combine multiple columns from multiple files into one file. Example file 1: c0t0d0 c0t2d0 # hostname vgname c0t0d1 c0t2d1 # hostname vgname c0t0d2 c0t2d2 # hostname vgname c0t1d0 c0t3d0 # hostname vgname1... (5 Replies)
Discussion started by: martva
5 Replies

2. Shell Programming and Scripting

Combine multiple string into 1 string group by certain criteria

Hi all, I am newbie in unix. Just have some doubts on how to join multiple lines into single line. I have 1 file with following contents. R96087641 HostName-kul480My This is no use any more %% E78343970 LocalPath-/app/usr/SG (Blank in this line) %% E73615740... (4 Replies)
Discussion started by: whchee
4 Replies

3. Shell Programming and Scripting

Combine common line from 2 Huge files

Hi, I am having 2 huge files having line count more than 10million. The files look like: File 1 45905099 2059 942961505 3007 8450875165 7007 615565331 3015 9415586035 9012 9871573 5367 4415655 4011 44415539519 5361 3250659295 4001 5950718618 9367 File 2 44415539519 TQ03... (2 Replies)
Discussion started by: rochitsharma
2 Replies

4. Shell Programming and Scripting

Get common lines from multiple files

FileA chr1 31237964 NP_001018494.1 PUM1 M340L chr1 31237964 NP_055491.1 PUM1 M340L chr1 33251518 NP_037543.1 AK2 H191D chr1 33251518 NP_001616.1 AK2 H191D chr1 57027345 NP_001004303.2 C1orf168 P270S FileB chr1 ... (9 Replies)
Discussion started by: genehunter
9 Replies

5. Shell Programming and Scripting

Need help joining two files with a common string

Hi all, I have one file that is in the form: S0243K05_T7_S0243K05_|_BASS2243.C7_K05 groupVI. 88.76 S0137F20_SP6_S0137F20_|_BASS2137d.SPB2.2_C10 groupXXI 88.06 S0056F03_T7_S0056F03_|_BASS256c.C7_C02 groupXIX 85.99 S0056F03_T7_S0056F03_|_BASS256c.C7_C02 groupXIX 83.23... (3 Replies)
Discussion started by: repiv
3 Replies

6. Emergency UNIX and Linux Support

Combine multiple Files into one big file

Hi Ppl, I have a requirement like i will be getting files of huge size daily and if the file size is so huge ,the files will be split into many parts and sent.The first file will have the header details followed by detail records and the consecutive files will have detail records and the last... (11 Replies)
Discussion started by: ganesh_248
11 Replies

7. Shell Programming and Scripting

Combine Multiple Files into Single One File One after other

I am trying to combine 4 .dat files into one single Output file Inputs are:- file123.dat, file256.dat, file378.dat & file490 Expected Output:- FileName=file1 {text from file1} EOF {blank line} FileName=file2 {text from file2} EOF {blank line} FileName=file3 {text from file3} EOF... (4 Replies)
Discussion started by: lancesunny
4 Replies

8. Shell Programming and Scripting

Compare multiple files, and extract items that are common to ALL files only

I have this code awk 'NR==FNR{a=$1;next} a' file1 file2 which does what I need it to do, but for only two files. I want to make it so that I can have multiple files (for example 30) and the code will return only the items that are in every single one of those files and ignore the ones... (7 Replies)
Discussion started by: castrojc
7 Replies

9. UNIX for Dummies Questions & Answers

Want to change common line from multiple files

Hi everyone, I've a requirement to modify an existing line which is common to multiple files. I need to replace that existing line with a new line. I've almost 900 ksh files to edit in the similar fashion in the same directory. Example: Existing Line: . $HOME/.eff.env (notice the "." at the... (3 Replies)
Discussion started by: kaleem.adil
3 Replies

10. Shell Programming and Scripting

Compare multiple files, identify common records and combine unique values into one file

Good morning all, I have a problem that is one step beyond a standard awk compare. I would like to compare three files which have several thousand records against a fourth file. All of them have a value in each row that is identical, and one value in each of those rows which may be duplicated... (1 Reply)
Discussion started by: nashton
1 Replies
IMAGEMATH(1)							   User Commands						      IMAGEMATH(1)

NAME
ImageMath - part of ANTS registration suite DESCRIPTION
Usage: ./ImageMath ImageDimension OutputImage.ext Operator Image1.ext Image2.extOrFloat some options output text files The last two arguments can be an image or float value Valid Operators : m (multiply) , + (add) , - (subtract) , / (divide) , ^ (power) , exp -- take exponent exp(imagevalue*value) addtozero overadd abs total -- sums up values in an image or in image1*image2 (img2 is the probability mask) Decision -- computes result=1./(1.+exp(-1.0*( pix1-0.25)/pix2)) Neg (Produce Image Negative ) , G Image1.ext s (Smooth with Gaussian of sigma = s ) MD Image1.ext s ( Morphological Dilation with radius s ) , ME Image1.ext s ( Morphological Erosion with radius s ) , MO Image1.ext s ( Morphological Opening with radius s ) MC Image1.ext ( Morphological Closing with radius s ) GD Image1.ext s ( Grayscale Dilation with radius s ) , GE Image1.ext s ( Grayscale Erosion with radius s ) , GO Image1.ext s ( Grayscale Opening with radius s ) GC Image1.ext ( Grayscale Closing with radius s ) D (DistanceTransform) , Segment Image1.ext N-Classes LocalityVsGlobalityWeight-In-ZeroToOneRange OptionalPriorImages ( Segment an Image with option of Priors , weight 1 => maximally local/prior-based ) Grad Image.ext S ( Gradient magnitude with sigma s -- if normalize, then output in range [0, 1] ) , Laplacian Image.ext S normalize? ( laplacian computed with sigma s -- if normalize, then output in range [0, 1] ) , Normalize image.ext opt ( Normalize to [0,1] option instead divides by average value ) PH (Print Header) , Byte ( Convert to Byte image in [0,255] ) LabelStats labelimage.ext valueimage.nii ( compute volumes / masses of objects in a label image -- write to text file ) ROIStatistics LabelNames.txt labelimage.ext valueimage.nii ( see the code ) DiceAndMinDistSum LabelImage1.ext LabelImage2.ext OptionalDistImage -- outputs DiceAndMinDistSum and Dice Overlap to text log file + optional dis- tance image Lipschitz VectorFieldName -- prints to cout & writes to image InvId VectorFieldName VectorFieldName -- prints to cout & writes to image GetLargestComponent InputImage {MinObjectSize} -- get largest object in image ThresholdAtMean Image %ofMean FlattenImage Image %ofMax -- replaces values greater than %ofMax*Max to the value %ofMax*Max stack Image1.nii.gz Image2.nii.gz --- will put these 2 images in the same volume CorruptImage Image NoiseLevel Smoothing TileImages NumColumns ImageList* RemoveLabelInterfaces ImageIn EnumerateLabelInterfaces ImageIn ColoredImageOutname NeighborFractionToIgnore FitSphere GM-ImageIn {WM-Image} {MaxRad-Default=5} HistogramMatch SourceImage ReferenceImage {NumberBins-Default=255} {Number- Points-Default=64} PadImage ImageIn Pad-Number ( if Pad-Number is negative, de-Padding occurs ) Where Image ValueToLookFor maskIm- age-option tolerance --- the where function from IDL TensorFA DTImage TensorColor DTImage --- produces RGB values identifying prin- cipal directions TensorToVector DTImage WhichVec --- produces vector field identifying one of the principal directions, 2 = largest eigenvalue TensorToVectorComponent DTImage WhichVec --- 0 => 2 produces component of the principal vector field , i.e. largest ei- genvalue. 3 = 8 => gets values from the tensor TensorIOTest DTImage --- will write the DT image back out ... tests I/O processes for consistency. MakeImage SizeX SizeY {SizeZ} SetOrGetPixel ImageIn Get/Set-Value IndexX IndexY {IndexZ} -- for example ImageMath 2 outimage.nii SetOrGetPixel Image Get 24 34 -- gets the value at 24, 34 ImageMath 2 outimage.nii SetOrGetPixel Image 1.e9 24 34 -- this sets 1.e9 as the value at 23 34 you can also pass a boolean at the end to force the physical space to be used TensorMeanDiffusion DTImage CompareHeadersAndImages Image1 Image2 --- tries to find and fix header error! output is the repaired image with new header. never use this if you trust your header information. CountVoxelDifference Image1 Image2 Mask --- the where function from IDL stack image1 image2 --- stack image2 onto image1 CorrelationUpdate Image1 Image2 RegionRadius --- in voxels , Compute update that makes Image2 more like Image1 ConvertImageToFile imagevalues.nii {Optional-ImageMask.nii} -- will write voxel values to a file PValueImage TValueImage dof ConvertToGaussian TValueImage sigma-float ConvertImageSetToMatrix rowcoloption Mask.nii *images.nii -- each row/column contains image content extracted from mask applied to images in *img.nii ConvertVector- ToImage Mask.nii vector.nii -- the vector contains image content extracted from a mask - here we return the vector to its spatial origins as image content TriPlanarView ImageIn.nii.gz PercentageToClampLowIntensity PercentageToClampHiIntensity x-slice y-slice z-slice TruncateImageIntensity inputImage {lowerQuantile=0.05} {upperQuantile=0.95} {numberOfBins=65} {binary-maskImage} Fill- Holes Image parameter : parameter = ratio of edge at object to edge at background = 1 is a definite hole bounded by object only, 0.99 is close -- default of parameter > 1 will fill all holes PropagateLabelsThroughMask speed/binaryimagemask.nii.gz initiallabelimage.nii.gz Optional-Stopping-Value -- final output is the propagated label image optional stopping value -- higher values allow more distant propagation FastMarchingSegmentation speed/binaryimagemask.nii.gz initiallabelimage.nii.gz Optional-Stopping-Value -- final output is the propagated label image optional stopping value -- higher values allow more distant propagation ExtractSlice volume.nii.gz slicetoextract --- will extract slice number from last dimension of volume (2,3,4) dimensions ConvertLandmarkFile InFile.txt ---- will convert landmark file between formats. see ants.pdf for description of formats. e.g. ImageMath 3 outfile.vtk ConvertLandmarkFile infile.txt ImageMath 1.9 May 2012 IMAGEMATH(1)
All times are GMT -4. The time now is 08:58 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy