Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

syncsort in shell script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting syncsort in shell script
# 1  
Old 11-15-2011
syncsort in shell script
hi ,
As of my understanding syncsort is an utility tool, that can perform sorting, merging, aggregation opertions, that can be uswd across platforms. using shell script we can call this syncsort. my qns is, what are the commands in syncsort say what does the following commands' function,
Code:
/CONDITION
/SUMMARIZE
/SUMMARIZE
/REFORMAT
/STATISTICS
/DERIVEDFIELD

consider this script, pls explain how it works,
Code:
#!/bin/ksh
syncsort <<___end_syncsort___
/WORKSPACE /appldata
/INFILE /appldata1/CIF_MTD_SUMM.dat 173
/FIELDS 
per_dt 1 char 10,
prod_id 11 char 6,
mkt_grp_id 17 char 6,
chan_id 23 char 6,
tenure_id 29 char 6,
cr_lmt_id 35 char 6,
bil_sys_cd 41 char 6,
sta_cd 47 char 6,
color_cd 53 char 1,
express_pay_id 54 char 6,
age_id 60 char 6,
glos_id 66 char 6,
role_id 72 char 6,
Basic_cif_count 78 EN 11 ,
Supp_cif_count 89 EN 11,
Cur_Charg_Am 100 EN 17 ,
Tot_Cr_Am 117 EN 17 , 
Tot_Bal_Am 134 EN 17 , 
Basic_Issue_Card_Cnt 151 EN 11,
Supp_Issue_Card_Cnt 162 EN 11 
/KEYS 
per_dt,
prod_id,
mkt_grp_id,
chan_id,
tenure_id,
cr_lmt_id,
bil_sys_cd,
sta_cd,
color_cd,
express_pay_id,
age_id,
glos_id,
role_id
/CONDITION
CHECKACTSTA (sta_cd GE "+00000" AND sta_cd LT "+25000") OR (sta_cd GT "+28000")
/CONDITION
CHECKCANSTA (sta_cd GT "+25000" AND sta_cd LT "+28000")
/SUMMARIZE
　
/OUTFILE /appldata/output/CIFACTIVE.dat OVERWRITE 
/INCLUDE CHECKACTSTA
/REFORMAT
per_dt,
prod_id,
mkt_grp_id,
chan_id,
tenure_id,
cr_lmt_id,
bil_sys_cd,
sta_cd,
color_cd,
express_pay_id,
age_id,
glos_id,
role_id,
Basic_cif_count,
Supp_cif_count,
Cur_Charg_Am,
Tot_Cr_Am,
Tot_Bal_Am,
Basic_Issue_Card_Cnt,
Supp_Issue_Card_Cnt
/STATISTICS
/OUTFILE /appldata/output/CIFCANCEL.dat OVERWRITE 
/INCLUDE CHECKCANSTA
/REFORMAT
per_dt,
prod_id,
mkt_grp_id,
chan_id,
tenure_id,
cr_lmt_id,
bil_sys_cd,
sta_cd,
color_cd,
express_pay_id,
age_id,
glos_id,
role_id,
Basic_cif_count,
Supp_cif_count,
Cur_Charg_Am,
Tot_Cr_Am,
Tot_Bal_Am,
Basic_Issue_Card_Cnt,
Supp_Issue_Card_Cnt
/STATISTICS
/END
___end_syncsort___
　


Moderator's Comments:
Mod Comment Please use code tags!
Last edited by zaxxon; 11-15-2011 at 05:59 AM.. Reason: code tags, see PM
# 2  
Old 11-15-2011
https://www.unix.com/shell-programmin...-syncsort.html

by DGPickett - I am not syncsort expert, and it is very proprietary about it's magic, so if you have a license, I bet they are a real good source of information. Also, look in your installation directories for man dirs, html pages and txt files.
# 3  
Old 11-16-2011
Hi,

The syntax you reference below and the functional description of each syntax option are all documented in the syncsort unix reference guide. If you are licensed customer you are entitled to a product manual. If you do not have access to one I can certainly assist you with this if you provide me with your contact information(full name, company name, support site contact)

Also, I wanted to let you know that the Syncsort syntax is proprietary and confidential and may not be posted on public forums such as this per the Syncsort License agreement. So if you have any specific product questions about syncsort unix product syntax I highly recommend you email our product support group directly. Our support email address is posted on our website.

I can describe at a high level what each of these options functional provide. These options are really documented well in the syncsort unix reference manual.

/CONDITION - allow for conditional field level or record level processing
/SUMMARIZE - removes duplicate records based on key values
/INCLUDE - allows for record selection and inclusion in the target output used with /condition
/REFORMAT - reformats and reorders fields with a record. This can be used with the /condition statement to logically include or not include data fie
/STATISTICS - produces operation metadata about the transformation, for example records read, records sorted, records output, elapsed time, cpu time.
/DERIVEDFIELD - derives new data, can be done conditionally as well based on other field values.
# 4  
Old 02-25-2012
Hi,

I have similar kind of question. So thought i will continue with this thread.

In my project we are calling Syncsort from shell script and we are using syncsort for sorting,merging and reformatting the fixed length binary format data.

The options of Syncsort which we are using are SORT, SUMMARIZE and REFORMAT.

/INFILE /Syncsortinput.dat 56
/RECORDLAYOUT
/KEYS
.....
/SUMMARIZE
....
/REFORMAT
....


Can someone explain with an example of different scenariors of SUMMARIZE and REFORMAT by taking some sample records ?
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. How to Post in the The UNIX and Linux Forums

How to get defined precision after arithmetic operation using syncsort?

I have to do some arithmetic operation on Field 8 which is calculated by Field 9/Field 7 Suppose i have data like : 0800123456|JAN|2017|JAN|2018|0800123456|0|0.0000|0.00| 0800234567|JAN|2017|JAN|2018|0800234567|4|2.5812|10.32| 0800666666|JAN|2017|JAN|2018|0800666666|2|1.7255|3.45|... (0 Replies)
Discussion started by: pumrao
0 Replies

2. UNIX for Advanced & Expert Users

Alternative open source to syncsort

Hi, I am looking for an opensource alternate to replace syncsort. Can you please suggest ? (8 Replies)
Discussion started by: AmbikaValagonda
8 Replies

3. Shell Programming and Scripting

calling syncsort in shell script

Hi, pls, tell me how to use syncsort in shell script? if i have to sort a file, what are the syncsort commands i hav to use ? say abc.dat is my file and dt, Id are my key columns. (0 Replies)
Discussion started by: captain haddock
0 Replies

4. UNIX for Dummies Questions & Answers

calling syncsort in shell script

Hi, pls, tell me how to use syncsort in shell script? if i have to sort a file, what are the syncsort commands i hav to use ? say abc.dat is my file and dt, Id are my key columns. (0 Replies)
Discussion started by: captain haddock
0 Replies

5. Shell Programming and Scripting

Syncsort Statistics - Work space used sometimes zero

I've searched previous forums but could not find the answer to my question. What determines how the "Work space used (bytes)" line item in the SyncSort Statistics appears? For instance I have some that show: Records read: 3,273,645 Data read (bytes): ... (2 Replies)
Discussion started by: mango
2 Replies

6. Shell Programming and Scripting

syncsort

I have got the following error messahe in the log file while running the syncsort process in a unix script. Starting the Syncsort Process SyncSort : (KEYEXPWARN) the license key (43966) will expire on Feb 28, 2011 (TECHSVC) please contact Syncsort Technical Support SyncSort... (3 Replies)
Discussion started by: mady135
3 Replies

7. Shell Programming and Scripting

Syncsort is getting killed

Hi All, I am running synsort utility via unix scripts. As soon as the control reaches the syncsort command,Its getting killed. The error message is /proj/cdw/syncsort/dev/copay/CdwWkRvrslFileReformat.syn: 8339696 Killed . Please suggest to overcome this issue. (5 Replies)
Discussion started by: prashantnandi
5 Replies

8. UNIX for Advanced & Expert Users

alternative open source to syncsort/dfsort ?

we use syncsort only as a sorting tool . Does anyone know about a free option for sort large binary files ? Thanks Golan (0 Replies)
Discussion started by: ghadad
0 Replies

9. Shell Programming and Scripting

syncsort error

I am getting the error Syncsort: (INERR) an internal error has occurred (11 in SSTRPHDL) when i run a syncsort. The manual does not give any explanation for this. Could anyone please give what the error is like. Thanks in advance (1 Reply)
Discussion started by: vinod.thayil
1 Replies
Login or Register to Ask a Question