Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Combine multiple Files into one big file
Homework and Emergencies Emergency UNIX and Linux Support Combine multiple Files into one big file Post 302573774 by Corona688 on Tuesday 15th of November 2011 01:27:56 PM
Old 11-15-2011
Code:
# List files in the current directory
ls |
# Feed it into the AWK programming language.
# splits each line apart $1,$2,...$NF on the _ character.
# Then puts it back together into STR, minus the _1,_2, etc on the end.
# Then stores it in T.
# Once it's read all lines, prints all unique combinations it found.
        awk -F_ '{ STR=$1; for(N=2; N<NF; N++) STR=STR"_"$N;  T[STR]=1 } END { for(K in T) print K; }' |
# Reads unique combinations from awk, like Daily_report_2222_20111115
# and puts them in the variable LINE.
# feed all files matching ${LINE}_* into cat, writing to the file $LINE.
        while read LINE
        do
                cat ${LINE}_* > $LINE
        done

These 2 Users Gave Thanks to Corona688 For This Post:
ganesh_248 Ion_Shell
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help Needed : Split one big file to multiple files

Hi friends, I have data in flat file as following, first filed is the customer number. We have almost 50-100 customers in the system 100 ABC A123 100 BVC D234 100 BNC N324 200 CBC A122 200 AVC D294 200 HNC N324 300 GBC A173 300 FVC D234 300 DNC N344 I want to split the file and... (5 Replies)
Discussion started by: monicasgupta
5 Replies

2. Shell Programming and Scripting

Combine multiple columns from multiple files

Hi there, I was wondering if someone can help me with this. I am trying the combine multiple columns from multiple files into one file. Example file 1: c0t0d0 c0t2d0 # hostname vgname c0t0d1 c0t2d1 # hostname vgname c0t0d2 c0t2d2 # hostname vgname c0t1d0 c0t3d0 # hostname vgname1... (5 Replies)
Discussion started by: martva
5 Replies

3. UNIX for Dummies Questions & Answers

Combine multiple files with common string into one new file.

I need to compile a large amount of data with a common string from individual text files throughout many directories. An example data file is below. I want to search for the following string, "cc_sectors_1" and combine all the data from each file which contains this string, into one new... (2 Replies)
Discussion started by: GradStudent2010
2 Replies

4. Shell Programming and Scripting

Combine Multiple text or csv files column-wise

Hi All I am trying to combine columns from multiple text files into a single file using paste command but the record length being unequal in the different files the data is running over to the closest empty cell on the left. Please see below. What can i do to resolve this ? File 1 File... (15 Replies)
Discussion started by: venky_ibm
15 Replies

5. Shell Programming and Scripting

combine multiple files by column into one files already sorted!

I have multiple files; each file contains a certain data in a column view simply i want to combine all those files into one file in columns example file1: a b c d file 2: 1 2 3 4 file 3: G (4 Replies)
Discussion started by: ahmedamro
4 Replies

6. Shell Programming and Scripting

Combine columns from multiple files

Can anybody help on the script to combine/concatenate columns from multiple files input1 4 135 5 185 6 85 11 30 16 72 17 30 21 52 22 76 input2 2 50 4 50 6 33 8 62 10 25 12 46 14 42 15 46output (2 Replies)
Discussion started by: sdf
2 Replies

7. Shell Programming and Scripting

Combine Multiple Files into Single One File One after other

I am trying to combine 4 .dat files into one single Output file Inputs are:- file123.dat, file256.dat, file378.dat & file490 Expected Output:- FileName=file1 {text from file1} EOF {blank line} FileName=file2 {text from file2} EOF {blank line} FileName=file3 {text from file3} EOF... (4 Replies)
Discussion started by: lancesunny
4 Replies

8. Shell Programming and Scripting

Split a big file into multiple files based on first four characters

I have a requirement to split a huge file to smaller text files based on first four characters which look like ABCD 1234 DFGH RREX : : : : : 0000 Each of these records are OF EQUAL bytes with a different internal layout based on the above first digit identifier.. Any help to start... (5 Replies)
Discussion started by: etldev
5 Replies

9. Shell Programming and Scripting

Compare multiple files, identify common records and combine unique values into one file

Good morning all, I have a problem that is one step beyond a standard awk compare. I would like to compare three files which have several thousand records against a fourth file. All of them have a value in each row that is identical, and one value in each of those rows which may be duplicated... (1 Reply)
Discussion started by: nashton
1 Replies

10. Shell Programming and Scripting

Split a big file into multiple files using awk

this thread is a continuation from previous thread https://www.unix.com/shell-programming-and-scripting/223901-split-big-file-into-multiple-files-based-first-four-characters.html ..I am using awk to split file and I have a syntax error while executing the below code I am using AIX 7.2... (4 Replies)
Discussion started by: etldev
4 Replies

LEARN ABOUT OSX

mecab

MECAB(1)							       MeCab								  MECAB(1)

NAME

       mecab - manual page for mecab of 0.95pre1

SYNOPSIS

       mecab [options] files

DESCRIPTION

       MeCab: Yet Another Part-of-Speech and Morphological Analyzer

       Copyright(C) 2001-2007 Taku Kudo Copyright(C) 2004-2006 Nippon Telegraph and Telephone Corporation

       -r, --rcfile=FILE
	      use FILE as resource file

       -d, --dicdir=DIR
	      set DIR  as a system dicdir

       -u, --userdic=FILE
	      use FILE as a user dictionary

       -l, --lattice-level=INT
	      lattice information level (default 0)

       -D, --dictionary-info
	      show dictionary information and exit

       -a, --all-morphs
	      output all morphs(default false)

       -O, --output-format-type=TYPE
	      set output format type (wakati,none,...)

       -p, --partial
	      partial parsing mode

       -F, --node-format=STR
	      use STR as the user-defined node format

       -U, --unk-format=STR
	      use STR as the user-defined unk format

       -B, --bos-format=STR
	      use STR as the user-defined bos format

       -E, --eos-format=STR
	      use STR as the user-defined eos format

       -x, --unk-feature=STR
	      use STR as the feature for unknown word

       -b, --input-buffer-size=INT
	      set input buffer size (default 8192)

       -P, --dump-config
	      dump MeCab parameters

       -C, --allocate-sentence
	      allocate new memory for input sentence

       -N, --nbest=INT
	      output N best results (default 1)

       -t, --theta=FLOAT
	      set temparature parameter theta (default 0.75)

       -o, --output=FILE
	      set the output file name

       -v, --version
	      show the version and exit.

       -h, --help
	      show this help and exit.

mecab of 0.95pre1						    March 2007								  MECAB(1)
All times are GMT -4. The time now is 04:49 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy