Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Complex data sorting in excel files or text files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Complex data sorting in excel files or text files
# 1  
Old 06-06-2012
Complex data sorting in excel files or text files
Dear all,
I have a complex data file shown below,,,,,

Code:
A_ABCD_13208   0   0   4.16735   141044   902449   1293900   168919   
C_ABCD_13208   0   0   4.16735   141044   902449   1293900   168919 
A_ABCDEF715   52410.9   18598.2   10611   10754.7   122535   252426   36631.4 
C_DBCDI_1353   0   26.512   0   93.9469   114151   94382.8   19043.1      
A_DBCDI_1353   0   26.512   0   93.9469   114151   94382.8   19043.1       
C_EFGH_24808   0   0   11.1129   5281.16   108786   146594   49778.1       
A_EFGH_9099   0   0   11.1129   5281.16   108786   146594   49778.1       
C_QRST_9938   0   0   0   2992.88   77887.8   60751.7   5253.41       
A_QRST_9938   0   0   0   2992.88   77887.8   60751.7   5253.41       
A_XVYZ_24808   0   0   0   33505.5   69088.4   167365   90621.9      
C_GHIH_9099   0   0   0   33505.5   69088.4   167365   90621.9      
C_TRST_7849   0   0   22.2259   2107.09   33073.1   42576.2   39891.2       
A_TRST_7849   0   0   22.2259   2107.09   33073.1   42576.2   39891.2       
A_ABCDI_15931   28.998   30.9306   11.1129   17966.2   32947.8   17405.4   3993.58       
A_ABCDI_15930   28.998   30.9306   11.1129   17966.2   32947.8   17405.4   3993.58      
C_GHJK_30564   0   0   0   214.736   30435.4   68135.6   69661.6       
A_GHJK_30564   0   0   0   214.736   30435.4   68135.6   69661.6       
C_STDT_2657   0   0   5.55647   1503.15   27929   101912   63628.2       
A_STDT_2657   0   0   5.55647   1503.15   27929   101912   63628.2

I want to sort to get following information

1: how many are pairs of A and C in the table? (A_ABCD_13208 and C_ABCD_13208)
2: How many only A and only C?
3: As I have many tables , how can I compare two tables for the unique ids in first column and for common ids??
4: Is there any software to create a Venn diagram for seven data sets???

Thanks a lot
Last edited by AAWT; 06-06-2012 at 10:19 AM..
# 2  
Old 06-06-2012
You asked many questions...
Here are some starts to your first couple questions:
Code:
$ cut -d" " -f1 <sample13.txt | cut -c2- | uniq -d
_ABCD_13208
_DBCDI_1353
_QRST_9938
_TRST_7849
_GHJK_30564
_STDT_2657

$ cut -d" " -f1 <sample13.txt | cut -c2- | uniq -u
_ABCDEF715
_EFGH_24808
_EFGH_9099
_XVYZ_24808
_GHIH_9099
_ABCDI_15931
_ABCDI_15930

The first example above shows the duplicated entries. The 2nd example shows unique lines. A 'wc' command could be appended to get a count.
# 3  
Old 06-06-2012
Hi Joeyg,

its not possible to get new file with A_ and C_ to be sure about pairs and unique.
this command give without A & C.

thanks
# 4  
Old 06-06-2012
what about something like:
Code:
$ cut -d" " -f1 <sample13.txt | sed 's/^[AC]/~/' | uniq -d
~_ABCD_13208
~_DBCDI_1353
~_QRST_9938
~_TRST_7849
~_GHJK_30564
~_STDT_2657

In this example, which would work for #1 and #2 on your list, I change first characters of A or C to ~.
# 5  
Old 06-08-2012
I am sorry but it is not giving me required result. now it is giving this sign
~ instead of A and C
# 6  
Old 06-08-2012
Code:
## A and C pair count ("A_" line and if next line "C_")
# awk '!/^ *$/{if(w==1){x1=xy;x2=$1}else{if($1)x1=$1;getline p;if(p)x2=substr(p,0,13);else next};if(x1~/^A_/){if(x2~/^C_/){c++;w=0}else{w=1;xy=x2}}}
END{print c}' file

## A record counts
# awk '/^A/{a[x++]=substr(p,0,1)}END{print x}' file

## C record counts
# awk '/^C/{a[x++]=substr(p,0,1)}END{print x}' file

## first column and counts with uniq
# awk '!/^ *$/{!f[$1]++}END{for(i in f)print i,f[i]}' file

regards
ygemici
This User Gave Thanks to ygemici For This Post:
AAWT
# 7  
Old 06-08-2012
Thanks a lot,,
but its only giving me counts like, 5432, 0r 2345

for A and C
is it possible to get new file with all A or C or A C pairs

Regards
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge and Sort tabular data from different text files

I have 42 text files; each containing up to 34 lines with following structure; file1 H-01 23 H-03 5 H-05 9 H-02 14 . . file2 H-01 17 H-02 43 H-04 7 H-05 8 H-03 7 . . file3 (6 Replies)
Discussion started by: Syeda Sumayya
6 Replies

2. UNIX for Dummies Questions & Answers

Unexplained text in data files

Has anyone ever encountered text from other files suddenly appearing in another data file that is not being used. There does not seem to be any reason for it, any thoughts would be useful. Thanks (14 Replies)
Discussion started by: SRoberts
14 Replies

3. Shell Programming and Scripting

Sorting indented text files

Hello, I'm trying to find a solution or a proper tool for the following job: I need to sort a text document with indented sections, so all levels of indentation are sorted independently for each section. Particularly, I need this for Cisco routers' running config files to compare them with... (2 Replies)
Discussion started by: kobel
2 Replies

4. Shell Programming and Scripting

Perl script to Merge contents of 2 different excel files in a single excel file

All, I have an excel sheet Excel1.xls that has some entries. I have one more excel sheet Excel2.xls that has entries only in those cells which are blank in Excel1.xls These may be in different workbooks. They are totally independent made by 2 different users. I have placed them in a... (1 Reply)
Discussion started by: Anamika08
1 Replies

5. Shell Programming and Scripting

How to write text file data to excel using UNIX shell script?

Hi All, I have the requirement in unix shell script. I want to write the "ls -ltr" command out put to excel file as below. Input :text file data : drwxr-xr-x 5 root root 4096 Oct 2 12:26 drwxr-xr-x 2 apx aim 4096 Nov 29 18:40 drwxr-xr-x 5 root root 4096 Oct 2 12:26 drwxr-xr-x... (10 Replies)
Discussion started by: Balasankar
10 Replies

6. Shell Programming and Scripting

Need help on inserting data from text file to excel using shell script

Hi, Please help me on this. I want to insert data from text file to excel using shell script nawk -v r=4 -v c=4 -v val=$a -F, 'BEGIN{OFS=","}; NR != r; NR == r {$c = val; print}' "file.csv" I used above one to insert $a value in 4th row, 4th column in an excel file.csv and it... (3 Replies)
Discussion started by: suman.frnz
3 Replies

7. Shell Programming and Scripting

Attaching two text files in two different sheet in same excel

Hi, My requirement is to get attach two different text file contents to two different sheets in same excelsheet. Also, is there any way we can name the tabs as desired ? Kindly assist. (2 Replies)
Discussion started by: sanjaydubey2006
2 Replies

8. Shell Programming and Scripting

sorting from several files for a specific data

Please assist: I have several files and all of the files have the same data format like following: All I need to get item next to "name" field and the "address" field from each file which has only 8 characters in "name" field. so the output should be: ams00ark(spcae)10.1.1.12... (3 Replies)
Discussion started by: amir07
3 Replies

9. Shell Programming and Scripting

PERL: Split Excel Workbook to Indiv Excel files

Hi, I am trying to find a way to read an excel work book with multiple worksheets. And write each worksheet into a new excel file using perl. My environment is Unix. For example: I have an excel workbook TEST.xls and it has Sheet1, Sheet2, Sheet3 worksheets. I would like to create... (2 Replies)
Discussion started by: sandeep78
2 Replies

10. UNIX for Dummies Questions & Answers

sorting files with find command before sending to text file

i need help with my script.... i am suppose to grab files within a certain date range now i have done that already using the touch and find command (found them in other threads) touch -d "$date_start" ./tmp1 touch -d "$date_end" ./tmp2 find "$data_location" -maxdepth 1 -newer ./tmp1 !... (6 Replies)
Discussion started by: deking
6 Replies
Login or Register to Ask a Question