Shell script/awk to sort text


 
Thread Tools Search this Thread
Homework and Emergencies Homework & Coursework Questions Shell script/awk to sort text
# 1  
Old 02-25-2014
Shell script/awk to sort text

1. The problem statement, all variables and given/known data:

I have a file with a fragment of a novel, which I have to clear from punctuation and sort all the words contained one per line and non duplicated, all this going to a file called "palabras".

Here is fragment of the input file:

Code:
Don Quijote de la Mancha, Cervantes 

Capítulo II

Que trata de la notable pendencia [*] que Sancho Panza tuvo con la sobrina y ama de don Quijote, con otros sujetos graciosos

And here is a fragment of how the file palabras should look like:

Code:
ama
Capítulo
Cervantes
con
Don
...

2. Relevant commands, code, scripts, algorithms:

3. The attempts at a solution (include all code and scripts):

Surfing on the web to find information, i have only achieved to clear punctuation and put a word in each line, with the following code:

Code:
{gsub("[-.,:;¿[\*\]\?]","");}
{RS=" ";}
{print > "palabras";}

calling it from terminal with this: cat novela | awk -f p4

p4 is the name of the file of my code.

and when i call from terminal this command: sort -u palabras>palabras2 it generates the file i want (if i put palabras>palabras it generates a blank file)

the question here is, how can i achieve my goal with in the same awk program? cuz i tried this:

Code:
{gsub("[-.,:;¿[\*\]\?]","");}
{RS=" ";}
{print > "palabras";}
END {sort -u > palabras2;}

With and without END, with sort -u > palabras2 and with sort -u palabras, however the file generated is the same without sorting and without deleting duplicated words.

I would really appreciate any ideas because I have been stucked on this problem for days. Also if you could suggest ideas, where i can call the awk like I said before ( cat novela | awk -f p4).

Thank you in advance.

4. Complete Name of School (University), City (State), Country, Name of Professor, and Course Number (Link to Course):

ITESM Campus Monterrey, Monterrey, Mexico
Profesor: Juan Jose Icaza
Course: Laboratorio de Sistemas Operativos
# 2  
Old 02-25-2014
You can use a pipeline (|) to feed the awk output into sort, both inside awk and outside awk using the shell. You do not need an END part.
# 3  
Old 02-25-2014
Hi guy,
Your awk version is important, don't know if your awk has a built in function 'asort'. If not, it will be hard for implement without 'sort'

Last edited by Lucas_0418; 02-25-2014 at 05:19 AM..
# 4  
Old 02-25-2014
Scrutinizer: Mmm could you provide me an example of a code to sort inside itself? Im new to this language because it is just a lab session of several lab sessions in a whole semester, so it would be very helpful.

And Lucas_0148, I dont really know the version, how could i get it? I just create a file and write inside it, and then execute it from terminal, in a fresh ubuntu installation.
# 5  
Old 02-25-2014
For example. Here sort is called as an external program from within awk
Code:
awk '{print | "sort"}' file

Here the output of awk is ran through sort through a pipeline..
Code:
awk '{print}' file | sort

This User Gave Thanks to Scrutinizer For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Shell script to call and sort awk script and output

I'm trying to create a shell script that takes a awk script that I wrote and a filename as an argument. I was able to get that done but I'm having trouble figuring out how to keep the header of the output at the top but sort the rest of the rows alphabetically. This is what I have now but it is... (1 Reply)
Discussion started by: Eric7giants
1 Replies

2. UNIX for Dummies Questions & Answers

How to sort a content of a text file using a shell script?

I am new to shell scripting. I am interested how to know how to sort a content of a file using shell scripting. I've attached the 'Input file' and the 'expected output' to this thread. Details provided in the expected output file will provide details on how the sort needs to be done. ... (16 Replies)
Discussion started by: nkarthik_mnnit
16 Replies

3. Shell Programming and Scripting

Creating new users using a text file as imput (using only shell script and/or awk)

I need somebody who could help with an exercise. You have a text file called users.txt with this info inside: users.txt: user1:1234:/home/homedir1 ; user2:1234:/home/homedir2 ; user3:1234:/home/homedir3 ; user4:1234:/home/homedir4 ; The script should create an user using the... (2 Replies)
Discussion started by: marcosruiz
2 Replies

4. Shell Programming and Scripting

How to sort the timestamp in the filename in shell script?

originally the shellscript #ln_file_name=`echo $ld_interface_date"_"${8}".csv"` #ln_file_name=`echo 201202011527_HL_HLTM1_B04A.csv` ln_file_name="*"`echo ${7}".csv"` get_file_list_1=$log_path"tm1_file_list.gfl1" cd ${source_path} echo "Try to find any file exist in the... (10 Replies)
Discussion started by: feilhk
10 Replies

5. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

6. Shell Programming and Scripting

AWK/Shell script for reading text file

Hello, I have a text file which has list of SQL Commands, from where I have grepped the lines where there is a specific string: grep <string> <file1> >> <file2> A sample of the file2 is: INSERT INTO PS_PWC_SP_ERN_DATA SELECT A.EMPLID ,B.COMPANY ,B.PAYGROUP , B.OTH_PAY FROM... (7 Replies)
Discussion started by: suddhasatwa_bha
7 Replies

7. Shell Programming and Scripting

shell script to sort information in one file

Hi to all, anyway to create shell script to sort informations from one file and create new file with the sorted values? from file 30days.out -bash-3.00# more 30days.out user/str4@kl.com/INBOX user/tg1@johor.com/INBOX user/tg2@kedah.com/INBOX user/tg3@titangroup.com/INBOX... (3 Replies)
Discussion started by: Mr_47
3 Replies

8. Shell Programming and Scripting

Text Proccessing with sort,uniq,awk

Hello, I have a log file with the following input: X , ID , Date, Time, Y 01,01368,2010-12-02,09:07:00,Pass 01,01368,2010-12-02,10:54:00,Pass 01,01368,2010-12-02,13:07:04,Pass 01,01368,2010-12-02,18:54:01,Pass 01,01368,2010-12-03,09:02:00,Pass 01,01368,2010-12-03,13:53:00,Pass... (12 Replies)
Discussion started by: rollyah
12 Replies

9. Shell Programming and Scripting

Unable to use sort command in Shell Script

Hello All, I am creating a shell script that reads a file(test.txt) with the following data, 0.0.0.0 10.10.10.0 10.10.10.1 10.10.10.10 10.10.10.2 10.10.10.3 10.10.10.4 10.10.10.5 10.10.10.6 10.10.10.7 10.10.10.8 10.10.10.9 If I use the sort, the highest value I am getting is... (1 Reply)
Discussion started by: racbern
1 Replies

10. Shell Programming and Scripting

Using sort with awk script

I have a file with four fields and an awk script that strips out one field displaying the remaining three. I have added headings for each of these fields such as Player - Year - RBIs then below it comes the data. What I am trying to do is sort the RBIs field in my script from most to least at the... (9 Replies)
Discussion started by: Trellot
9 Replies
Login or Register to Ask a Question