Identifying columns and their values based on schema file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Identifying columns and their values based on schema file
# 1  
Old 11-07-2013
Identifying columns and their values based on schema file

I have 3 files, data file,schema file and a threshold file.
Data file contains data in which columns are distributed according to schema file. This data file doesn't contain any headers.

Three continuous columns in the data file represent single variable in schema file.
first column represent new value,second column old value and third difference between 2 columns.

Is there a way to extract each variable value(all 3 columns) based on the schema file.

Let me explain the scenario with an example

Schema file

Code:
cust_id,Alphanumeric,99999999999999,1,14,22
Var1,Numeric,+999999999,1,10,36,V
Var2,Numeric,+999999999,1,10,46,V
Var3,Numeric,+999999999,1,10,56,V

Threshold file
Code:
GLOBAL,A,0
Var1,A,0.4
Var3,A,0.06

PFA the sample data pasted in excel. please ignore the headers,as it is mentioned only for better understanding.

Now i need to extract the variable values from data file where difference is not equal to zero and greater than threshold value into separate files with the header name as the output file name.

Last edited by Don Cragun; 11-07-2013 at 02:24 AM.. Reason: Change ICODE tags to multiple CODE tags.
# 2  
Old 11-07-2013
your requiremnet is not clear. please explain with more sample data and also sample of actual data file.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Linux

To get all the columns in a CSV file based on unique values of particular column

cat sample.csv ID,Name,no 1,AAA,1 2,BBB,1 3,AAA,1 4,BBB,1 cut -d',' -f2 sample.csv | sort | uniq this gives only the 2nd column values Name AAA BBB How to I get all the columns of CSV along with this? (1 Reply)
Discussion started by: sanvel
1 Replies

2. UNIX for Dummies Questions & Answers

Extracting rows from a text file based on the values of two columns (given ranges)

Hi, I have a tab delimited text file with multiple columns. The second and third columns include numbers that have not been sorted. I want to extract rows where the second column includes a value between -0.01 and 0.01 (including both numbers) and the first third column includes a value between... (1 Reply)
Discussion started by: evelibertine
1 Replies

3. Shell Programming and Scripting

Selecting lowest and highest values in columns 1 and 2, based on subsets in column 3

Hi, I have a file with the following columns: 361459 447394 CHL1 290282 290282 CHL1 361459 447394 CHL1 361459 447394 CHL1 178352861 178363529 AGA 178352861 178363529 AGA 178363657 178363657 AGA Essentially, using CHL1 as an example. For any line that has CHL1 in... (2 Replies)
Discussion started by: hubleo
2 Replies

4. Shell Programming and Scripting

Identifying entries based on 2 fields in a string.

Hi Guys, I’m struggling to use two fields to do a duplicate/ unique by output. I want to look IP addresses assigned to more than one account during a given period in the logs. So duplicate IP and account > 1 then print all the logs for that IP. I have been Using AWK (just as its installed... (3 Replies)
Discussion started by: wabbit02
3 Replies

5. Cybersecurity

LDAP rfc2307bis.schema or nis.schema?

Hi guys, we are implementing new ldap in our organization exclusively for Linux authentication purposes. As we are new to the subject, I would appreciate help regarding which schema to use to create group of users and accounts / passwords entries? I see both schemas listed in subject above... (0 Replies)
Discussion started by: Lastminute
0 Replies

6. UNIX for Dummies Questions & Answers

Removing columns from a text file that do not have any values in second and third columns

I have a text file that has three columns. But at the end of the text file, there are trailing lines that have missing second and third columns: 4 0.04972604 KLHL28 4 0.0497332 CSTB 4 0.04979822 AIF1 4 0.04983331 DECR2 4 0.04990344 KATNB1 4 4 4 4 How can I remove the trailing... (3 Replies)
Discussion started by: evelibertine
3 Replies

7. Shell Programming and Scripting

Selecting rows based on values in columns

Hi My pipe delimited .txt file contains rows with 10 columns. Can anyone advise how I output to file only those rows with the letters ‘ci' as the first 2 characters in the 3rd column ? Many thanks (4 Replies)
Discussion started by: malts18
4 Replies

8. Shell Programming and Scripting

Replacing values in a file based on values in another file

Hi I have 2 files:- 1. List of files which consists of names of some output files. 2. A delimited file; delimted by "|" I want to replace the value of the $23 (23rd column) in the delimited file with name in the first file. It is always position to position. Meaning first row of the first... (5 Replies)
Discussion started by: pparthiv
5 Replies

9. Shell Programming and Scripting

identifying null values in a file

I have a huge file with 20 fileds in each record and each field is seperated by "|". If i want to get all the reocrds that have 18th or for that matter any filed as null how can i do it? Please let me know (3 Replies)
Discussion started by: dsravan
3 Replies
Login or Register to Ask a Question