awk : split file and rename and save in path according to content
Hello,
I'm using Windows 7 ; sed, awk and gnuwin32 are installed.
I have a big text file I need to manipulate.
In short, I will have to split it in thousands of short files, then rename and save in a folder which name is based upon filename.
Here is a snippet of my big input.txt file (this pattern is repeated 70k times) :
As of now, I know how to split it with awk :
Yet I dont know how to drive awk in renaming files.
In every splitted file, there is a line (line number 3 of the splitted file) which says the name of the expected file. In showed snippet, it is : name=Bob.2111.
I expect to have about 70k files named like Bob.[0-9]+ (no extension).
My question number one is : is awk able to do that ? is awk able to catch a value in current file according to some pattern for renaming the splitted files ? I did not find anwser yet. Most examples order awk to use a name which doesnt come from the manipulated file.
My question number two is : is awk also able to save the splitted files in a constructed path, according to content ?
I want all splitted files containing Bob.1[0-9]+ to be saved in folder 1/ ; all Bob.2[0-9]+ to be saved in folder 2/, etc., for limiting the quantity of files per folder (using first number of numerical id).
I guess I need to catch every Bob.[0-9]+ in a variable $1.
Thus I may use this variable for (1) renaming files and (2) for moving files
If Bob.2111 is $1, I would like to save file like that {substring $1,4,20}/$1 which would give 2/Bob.2111.
As you can guess, I'm not expert
Thank you.
Last edited by sellig; 03-07-2015 at 01:18 PM..
Reason: typo
I'm not sure I grasped everything that you specified...
Why rename when you can save it immediately to the desired file? This requires all the directories exist:
Big input file is full of text, structured like that :
I need to split this big file in short files, from [ctime to ]ctime.
Which I can do using awk.
I'm looking for a way to tell awk (or else ?) how to name these splitted files. I wish it is doable with awk during the splitting step (split and give name).
I want files to be named depending to their content : Bob.2112, Bob.2115, Bob.2116, etc.
If we imagine the big input text has this form :
a=aa
b=bb
c=cc
d=dd
a=aa2
b=bb2
c=cc2
and value wanted is after c=, might it help for building the command ?
@Rudi : I'm trying to play with your code (on Windows, sorry), with no success atm : awk: (FILENAME=input.txt FNR=2) fatal : can't redirect to `/' (Permission denied)
Last edited by sellig; 03-07-2015 at 04:53 PM..
Reason: typo
This is the result for me:
Print the internal, new file name to stdout for debugging purposes.
And, why do you change the input file structure between posts?
---------- Post updated at 23:29 ---------- Previous update was at 23:04 ----------
To be able to handle maaany files, you might want to close (FN) just before you define the new one...
Hi,
Right now there is a file called 'qm.ini' which is owned by mqm:mqm and I am trying to replace a line from this file with something else and save.
I am using the below perl command to replace and save within a shell script with a different user called 'mqadm' which is also part of mqm... (1 Reply)
Hi All,
When i use sort Test, here is the output:
$ sort Test
a
b
b
c
d
e
f
g
h
i
But the contents in the file remain unsorted, how to do that?
]$ cat Test
g
i (6 Replies)
Hi
I have a large text file and I want to split its content into multiple flies.
this large file contains several blocks of codes separated by a comment line for each block.
this comment line represents a directory path
So, when separate these blocks each into a separate file, This output... (7 Replies)
Arun kumar something somehting Enterting in to the line
.
.
.
.
Some text text Finshing the sentence
Some other text
.
.
.
.
Again something somehting Enterting in to the line
.
.
.
.
.
.
Again text text Finshing the sentence (6 Replies)
Hi All;
I have input file like below
name char(3)
number number(3)
inputfile
namenumber
xyz123abc509kai330
aca203
ald390afa000als303
I wanted to split like below:-
output like this:-
xyz123
abc509
kai330
aca203
ald390 (6 Replies)
I have a file test1.html like below:
<dctm_topnav_en_US>
<html>
.....
</html>
<dctm_topnav_en_CA>
<html>
.....
</html>
<dctm_topnav_en_FR>
<html>
.....
</html>
I need to use awk to split this into three file names like en_US.html ,
en_CA.html, en_FR.html each having content between... (4 Replies)
Hi All,
I have a requirement .I want to split a file and the split files should have certain names.
Currently when i use the split command
split -1000 testdata testdata_
Then the output is
testdata_aa
testdata_bb
testdata_cc
and so on.
But i want the output as
testdata1.snd... (3 Replies)
Hi,
I have a file which contains records of data.
I need to split the file into multiple files depending upon the value of last field.
How do i read the last field of each record in the file???
Regards,
Chaitrali (4 Replies)