awk processing / Shell Script Processing to remove columns text file


 
Thread Tools Search this Thread
Top Forums Programming awk processing / Shell Script Processing to remove columns text file
# 1  
Old 03-23-2013
awk processing / Shell Script Processing to remove columns text file

Hello,

I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this :

This is the output of ls command : I stored the output in a file filelist
Code:
 1.1M AAAI94-018.pdf
 188K Acoustic instruments.pdf
1000K Ground-Truth Transcriptions .pdf

I would like to just get the filenames, so I need to filter this output . I tried the cut command like this :

Code:
cat filelist | cut -f 1 -d ' '

but it does not work because of varying file sizes. like the last line of the output does not have a space in the beginning.

Also problematic is that there are spaces within the filenames..

So I need to remove the first 7 columns of every line in this output. that will just give me the full name of each file.
How do I do it ?
# 2  
Old 03-23-2013
Here's one way using awk:
Code:
awk '{print substr($0,7,length($0)-6)}' filelist

Output:
Code:
AAAI94-018.pdf
Acoustic instruments.pdf
Ground-Truth Transcriptions .pdf

Or if you want to stick with cut command:
Code:
cut -c7- filelist


Last edited by mjf; 03-23-2013 at 10:56 AM..
# 3  
Old 03-23-2013
Code:
cut -c/- filelist

? (UUOC)
# 4  
Old 03-23-2013
Another approach:
Code:
awk '{sub($1,x);$1=$1}1' filelist

# 5  
Old 03-23-2013
Thanks

Hello,

Thank you for helping me out. It works..Smilie

---------- Post updated at 07:03 PM ---------- Previous update was at 06:00 PM ----------

Hello,

So I ran this Awk code in the end

Code:
awk '{print substr($0,7)}' filelist

through the shell like this : I wanted the output to get redirected to the same file as the input
thus,

Code:
[milli@milli ~]$ sh test.awk > filelist

I expected to see the filelist overwritten with the new input from the Results of the Shell Script, however, it produces a blank file.

If I give this command
Code:
[milli@milli ~]$ sh test.awk >> filelist

Then it gets appended as usual.

If I try a different filename for output redirection , then it works properly.

Any idea what is happening ?
# 6  
Old 03-23-2013
When shell encounters a redirection operator > the file will be truncated first (0 byte size) and then opened for writing, so after the redirection file is opened for writing, then shell starts test.awk process. This is the reason why you get a blank output file.

But in case of append >> the previous data is retained and then shell starts test.awk process. This is the reason why the data gets appended.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk for text processing

Hi,my file is in this format ", \"symbol\": \"Rbm38\" } ]" I want to convert it to a more user readable format _id pubmed text symbol 67196 18667844 Overexpression of UBE2T in NIH3T3 cells significantly promoted colony formation in mouse cell cultures Ube2t 56190 21764855 ... (3 Replies)
Discussion started by: biofreek
3 Replies

2. Shell Programming and Scripting

Text columns processing using awk

P { margin-bottom: 0.25cm; line-height: 120%; }CODE.cjk { font-family: "WenQuanYi Micro Hei",monospace; }CODE.ctl { font-family: "Lohit Hindi",monospace; }A:link { } I'm trying to build an awk statement to print from a file (file1): A 1,2,3 * A 4,5,6 ** B 1 ... (4 Replies)
Discussion started by: dovah
4 Replies

3. Shell Programming and Scripting

Text processing using awk

I dispose of two tab-delimited files (the first column is the primary key): File 1 (there are multiple rows sharing the same key, I cannot merge them) A 28,29,30,31 A 17,18,19 B 11,13,14,15 B 8,9File 2 (there is one only row beginning with a given key) A 2,8,18,30,31 B ... (3 Replies)
Discussion started by: dovah
3 Replies

4. Shell Programming and Scripting

AWK: Remove spaces before processing each line?

Hi, all I have a file containing the following data: name: PRODUCT_1 date: 2010-01-07 really_long_name: PRODUCT_ABCDEFG I want to get the date (it is "2010-01-07" here), I could use the following code to do that: awk... (6 Replies)
Discussion started by: kevintse
6 Replies

5. Shell Programming and Scripting

File Processing in shell script

i have a file with following type of data abcd : gggggg gggggg ; 1234 gggggg ; 5678 gggggg ; 3434 gggggg ; 6565 gggggg ; 1231 1234 ; vvvv ;Eng=Myfirstname 5678 ; xyzf ;Eng=Mysecondname 3434 ; xyzf ;Eng=Mythirdname 6565 ; xyzf ;Eng=Mysfourthname 1231 ; xyzf ;Eng=Mysfifthname... (7 Replies)
Discussion started by: telangmadhuri
7 Replies

6. Shell Programming and Scripting

Awk text processing

Hi Very much appreciate if somebody could give me a clue .. I undestand that it could be done with awk but have a limited experience. I have the following text in the file 1 909 YES NO 2 500 No NO . ... 1 ... (8 Replies)
Discussion started by: zam
8 Replies

7. Shell Programming and Scripting

2 file processing script in C shell

I want to use an awk for the following scenario but not sure if it will work or not. I have two input file: F1 and F2 F1 02 05 08 F2 00 01 02 03 04 05 06 07 08 09 10 (1 Reply)
Discussion started by: jclanc8
1 Replies

8. Shell Programming and Scripting

awk, perl Script for processing a single line text file

I need a script to process a huge single line text file: The sample of the text is: "forward_inline_item": "Inline", "options_region_Australia": "Australia", "server_event_err_msg": "There was an error attempting to save", "Token": "Yes", "family": "Family","pwd_login_tab": "Enter Your... (1 Reply)
Discussion started by: hmsadiq
1 Replies

9. Shell Programming and Scripting

KSH script -text file processing NULL issues

I'm trying to strip any garbage that may be at the end of my text file and that part is working. The problem only seems to be with the really long lines in the file. When the head command is executed I am directing the output to a new file. The new file always get a null in the 4096 position but... (2 Replies)
Discussion started by: geauxsaints
2 Replies

10. Shell Programming and Scripting

text processing ( sed/awk)

hi.. I have a file having record on in 1 line.... I want every 400 characters in a new line... means in 1st line 1-400 in 2nd line - 401-800 etc pl help. (12 Replies)
Discussion started by: clx
12 Replies
Login or Register to Ask a Question