Help with extracting data and plotting


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Help with extracting data and plotting
# 1  
Old 04-03-2009
Help with extracting data and plotting

I have attached a txt file, what I would like to be able to do is:
1. Extract Data from Columns labeled E/N and Ko into a new file
2. Then in the new file I would like to be able to plot E/N on the X axis and Ko on the y axis.
3. Lastly I would like to be able to extract multiple data sets and do this in one file so i can plot all my curves and see how they relate to each other.

Thank you as always for your help, you guys are great. Lenny.
# 2  
Old 04-03-2009
what have you tried so far?
# 3  
Old 04-03-2009
well nothing

I dont really know what i am doing. I am a chemist, so I would appreciate any help. Lenny
# 4  
Old 04-03-2009
I'll assume you're on a UNIX system and Perl and gnuplot are available.

The following (cheap and pretty dirty) Perl script will extract the two columns you're interested in:


Code:
#!/usr/bin/perl
#
#  Name: extract
#  Use: extract data.file
#
if ( $#ARGV == 0 )
  {              # First comand line arg is data file name
    $input_data = $ARGV[0];
    $output_data = $input_data . ".extracted_data";
  }
else
  {
    printf STDERR "Usage: extract data-file-name\n";
    exit 1;
  }

$extract_data = 0;   # Initially "false"

open(OUTPUT, '>', $output_data);

open (DATA, '<', $input_data);

RECORD:
while (<DATA>)
  {
    chomp;
    s/^\s+//;                # Remove leading whitespace
    if ( length($_) <= 0 )
      {       # Skipping empty lines
	next RECORD;
      }
    if ( /E\/N/ )
      {
	if ( / Ko / )
	  {             # Found end of the header records. Data follows...
	    $extract_data = -1;
	    next RECORD;
	  }
      }
    if ( $extract_data )
      {
	@values = split(/\s+/, $_);
	printf OUTPUT "%s %s\n", $values[0], $values[1];
      }
  }

close(DATA);

close(OUTPUT);

exit;

(Sorry... the forum software does awful things to the indentation of the script and I'm not sure how to prevent that.)

This would be run using:

Code:
   chmod u+x extract
   ./extract data.file

and produce

Code:
   data.file.extracted_data

A simple gnuplot command file to plot two curves might be:

set terminal postscript landscape;
plot "data.file.extracted_data" with lines, "data2.file.extracted_data" with lines;

If these were saved in a file named "gnuplot.cmds", you would generate the plot using:
Code:
gnuplot < gnuplot.cmds > plotted_data.ps

If you need to plot more than two curves, add another datafile name containing extracted data file to the end of the "plot" command and rerun the above gnuplot command. You could also add extra commands to control the X and Y axes, etc. (above the line containing the "plot" command).

Hope this helps.

--
Rick

Last edited by Yogesh Sawant; 04-04-2009 at 07:11 AM.. Reason: added code tags
# 5  
Old 04-04-2009
If you've got Windows and Microsoft Excel, you can open it with that and use the data to columns wizard to put your data in columns, then create the graph.
# 6  
Old 04-21-2009
how do i extract the data easily into excel ztodd. thanks everyone.
# 7  
Old 04-21-2009
I extracted the data into Excel for you- here you go.
If you wanna know how I did it, I did it by (I'm using Excel 2003):
1. copying your data starting with E/N Ko etc. and pasting it into a new Excel workbook,
2. then click the little paste button to see options, and click Use Text Import Wizard
3. Choose fixed width, click next
4. It shows lines between the columns- one of the lines was a little off, I adjusted it (click and drag the line)
5. Click finish
6. Now you can insert a char- click Insert, Chart
7. Choose a scatter plot. Take it from there!

I don't have a lot of experience with charts, etc. in Excel, but see how easy and user-friendly it is with Windows and Microsoft Office, compared to Unix!
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Fun with terminal plotting JSON data at the command line

One of the great thing about unix is the ability to pipe multiple programs together to manipulate data. Plain, unstructured text is the most common type of data that is passed between programs, but these days JSON is becoming more popular. I thought it would be fun to pipe together some command... (1 Reply)
Discussion started by: kbrazil
1 Replies

2. Shell Programming and Scripting

Help with extracting data within parentheses

This is my input file: a|b|c(ef)|g|h(km)|p My output file should look like: a|b|ef|g|km|p That is, pipe is the delimiter. The data within pipe must be displayed as it is but if it encounters any data within parentheses, then only the data within parentheses has to be displayed ( the data... (2 Replies)
Discussion started by: ksatish89
2 Replies

3. Shell Programming and Scripting

extracting data

I have a txt file of the following format >ab_ qwerty >rt_ hfjkil >Ty2 hglashglkasghkf; >P2 aklhfklflkkgfgkfl >ui_ vnllkdskkkffkfkkf >we32 vksksjksj;lslsf'sk's's .... ..... I want to split this big file based on the header (>) (5 Replies)
Discussion started by: Lucky Ali
5 Replies

4. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

5. Shell Programming and Scripting

extracting data from a string

Hi there, I have a bunch of vlan tagged network interfaces that are named as follows e1000g111000 e1000g99001 e1000g3456000 nge2002 where the 'e1000g' and 'nge' parts of the name are the driver, the red and blue bits above define the VLAN and the last digit on the end defines the... (3 Replies)
Discussion started by: rethink
3 Replies

6. UNIX for Dummies Questions & Answers

Extracting Data Using SED

Given the following text in a file named extract.txt: listenPort:=25 smtpDestination:=2 enableSSL:= I am trying to extract only the value 2 following smtpDestination:= Someone had suggested I use: sed -e "s/^smtpDestination:=\(.*\)$/\1/" extract.txt but this returns: listenPort:=25 2 ... (2 Replies)
Discussion started by: cleanden
2 Replies

7. UNIX for Dummies Questions & Answers

Plotting Data within UNIX

I have a set of data that looks similar to the following in UNIX: 0.12_0.008_fall_ff.out:bisect return: 0.08056640625 0.12_0.04_fall_ff.out:bisect return: 0.07470703125 0.12_0.12_fall_ff.out:bisect return: 0.06298828125 0.12_0.24_fall_ff.out:bisect return: 0.05126953125 Previously I have... (4 Replies)
Discussion started by: EDALBNUG
4 Replies

8. Shell Programming and Scripting

Extracting ICMP Output for Plotting

Dear, I want to perform a plotting using xgraph, and the plotting data (ping.txt) is as below. For the graph I just want to plot the time for x-axis (line count) and the RTT for y-axis (time in ms). Below are script i write for that purpose but it seen did not work. Any guide for me because i... (1 Reply)
Discussion started by: Paris Heng
1 Replies

9. UNIX for Advanced & Expert Users

Gnuplot question: plotting 3D data in map view

I have a simple gnuplot question. I have a set of points (list of x,y,z values; irregularly spaced, i.e. no grid) that I want to plot. I want the plot to look like this: - map view (no 3D view) - color of each point should depend on z-value. - I want to define my own color scale - plot should... (1 Reply)
Discussion started by: karman
1 Replies

10. Shell Programming and Scripting

Extracting certain data from a sentence

How do I delete text in each line from the first character up to a certain pattern, ie. 'qmuser.' and then delete from the next occurence of a dot to the end of the sentence For example: - LTSB Renewal Notice Travel Pack --- d \qmaster\spool1\qmuser.8664_LM245_20031216094308.ps.0 From this... (7 Replies)
Discussion started by: dbrundrett
7 Replies
Login or Register to Ask a Question