Sponsored Content
Top Forums Shell Programming and Scripting Need to prepare a CSV table with inputs from multiple files Post 303040668 by RudiC on Sunday 3rd of November 2019 04:31:44 PM
Old 11-03-2019
NA was not specified. Try
Code:
awk -F/ '
NR == 1         {printf "Sl#,VideoName,BitRate"
                }
FNR == NR       {PRM[NR] =  "( |no-)" $0 "=*[^ /]*"
                 MX = NR
                 printf ",%s", $0
                 next
                }
!HDFIN          {printf RS
                 HDFIN = 1
                }
FNR == 1        {gsub (" ", "", $3)
                 printf "%d,%s,%d", ++FCNT, FILENAME, $3+0
                }
FNR == 3        {for (i=1; i<=MX; i++)  {match ($0, PRM[i])
                                         n = split (substr ($0, RSTART, RLENGTH), TMP, "=")
                                         if (n == 2)            printf ",%s", TMP[2]
                                           else if (n == 1)     printf ",%d", (! (TMP[1] ~ /^no/))
                                                  else          printf ",NA"
                                        }
                 printf RS
                }

' Required_para.txt OFS=, vid?
Sl#,VideoName,BitRate,input-res,crf,aq-mode,strong-intra-smoothing,bframes,rc-lookahead,me,subme,merange,deblock,selective-sao,psy-rd,psy-rdoq,ctu,rdoq-level,max-merge,ref,max-tu-size,transfer,colormatrix,colorprim,selective-sao
1,vid1,5750,1920x816,23.3,3,0,8,100,3,7,92,0:0,NA,2.00,1.00,32,2,4,5,16,1,1,1,NA
2,vid2,3711,1920x804,18.0,3,1,8,25,3,3,57,0:0,0,2.00,1.00,64,2,3,4,32,2,2,2,0
3,vid3,7903,1920x800,18.0,3,1,8,25,3,3,57,0:0,0,2.00,1.00,64,2,3,4,32,2,2,2,0


No easy way. Read and exercise. What I did to learn awk is solve problems e.g. found in these fora and analyse / understand solutions given, and use the man page. I have to admit I have a history in other programming languages so wasn't a true novice.
This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to Pull out multiple files from DB table and redirect all those files to a differetn directory?

Hi everyone!! I have a database table, which has file_name as one of its fields. Example: File_ID File_Name Directory Size 0001 UNO_1232 /apps/opt 234 0002 UNO_1234 /apps/opt 788 0003 UNO_1235 /apps/opt 897 0004 UNO_1236 /apps/opt 568 I have to... (3 Replies)
Discussion started by: ss3944
3 Replies

2. Shell Programming and Scripting

Copying multiple csv files

Hi, I have mutiple csv files at server1 at /apps/test/data. I needed a script that would copy these csv files from server1 at /usr/data, put them in server2,archive the earlier files that were present in server2 before removing those already present. Kindly help. (2 Replies)
Discussion started by: Alok Ranjan
2 Replies

3. Shell Programming and Scripting

Field validations in multiple files CSV

Hi, I am regular reader of this forum. My advanced thanks to everyone. Below given are the sample files INDATA (Main data) Fild1Çfld2Çfld3….. Fild1Çfld2Çfld3….. Fild1Çfld2Çfld3….. Fild1Çfld2Çfld3….. Fild1Çfld2Çfld3….. . . N records (140000) eg GRPDATA (Reference file) (2 Replies)
Discussion started by: hyperion.krish
2 Replies

4. Shell Programming and Scripting

reading information from a table and apply a command on multiple files

Hey gyuz, I wanna calculate the number of mapped reads of a bam file in a region of interest. I used this code to do so : samtools view input.bam chrname:region1 > region1.txt This will store all the reads from given bam file within the region of interest in region1.txt Now I have... (5 Replies)
Discussion started by: @man
5 Replies

5. Shell Programming and Scripting

Find values in multiple csv files

Hi, I'd like to find the values of certain fields in multiple csv files stored in 1 directory based upon an input search string. An fgrep returns the complete record, I only want certain fields. Thanks in advance for your help. Perry (6 Replies)
Discussion started by: biscayne
6 Replies

6. Shell Programming and Scripting

Split a .csv File into Multiple Files

Hi guys, I have a requirement where i need to split a .csv file into multiple files. Say for example i have data.csv file and i have splitted that into multiple files based on some conditions i.e first file should have 100, last file 50 and other files 1000 each. Am passing the values in... (2 Replies)
Discussion started by: azherkn3
2 Replies

7. Shell Programming and Scripting

Insterting column in csv from multiple files

Hello, I have a spec file that contains a lot of strings that looks like this: PC DELL OptiPlex 3010MT i3 3220/2GB/500GB/DVD-RW/FREE DOS / 5Y NBD Intel i3 3220 (Dual Core, 3.30GHz, 3MB, w/ HD2500 Graphics), 2GB (1x2GB) DDR3 PC3-1600MHz, 500GB HDD SATA III 7200rpm, DVD+/-RW (16x),... (9 Replies)
Discussion started by: g9100
9 Replies

8. UNIX for Dummies Questions & Answers

Load multiple files into a table

Hi, I need to load data from two files to a single table. My requirement is that I get two files in which a few column data are manadatory. These files are identified based on the file name. For example, I have two files ABCFile and BCDFile. ABCFile has mandatory data in column 3 and 4... (0 Replies)
Discussion started by: reshma15193
0 Replies

9. Shell Programming and Scripting

CSV joining and checking multiple files

Hello, For our work we use several scripts to gather/combine data for use in our webshop. Untill now we did not had any problems but since a couple days we noticed some mismatches between imports. It happened that several barcodes where matched even though it was a complete other product. Of... (19 Replies)
Discussion started by: SDohmen
19 Replies

10. UNIX for Beginners Questions & Answers

Export Oracle multiple tables to multiple csv files using UNIX shell scripting

Hello All, just wanted to export multiple tables from oracle sql using unix shell script to csv file and the below code is exporting only the first table. Can you please suggest why? or any better idea? export FILE="/abc/autom/file/geo_JOB.csv" Export= `sqlplus -s dev01/password@dEV3... (16 Replies)
Discussion started by: Hope
16 Replies
AMC-EXPORT(1)						       Auto Multiple Choice						     AMC-EXPORT(1)

NAME
AMC-export - exports marks for AMC multiple choice exams. SYNOPSIS
auto-multiple-choice export --data project-data-dir --module module --fich-noms students-list.csv [--noms-encodage list-encoding] --o output-file DESCRIPTION
The AMC-export.pl command exports marks from an Auto Multiple Choice exam. --data project-data-dir gives the directory where data files are (see for example AMC-prepare(1)). --module module selects a module for export. See below for the modules included in the AMC standard distribution. --fich-noms students-list.csv sets the students list file. --noms-encodage list-encoding selects an encoding for file students-list.csv (default is utf-8). --o output-file gives the output file name. --option-out options gives an option for selected module, in the form key=value (see below for possible options for each module). To specify multiple options, use --option several times. --sort sort-type sort the students names, depending on sort-type. If sort-type is l, use line number from students list file to sort. If sort-type is m, use mark (and name if marks are equal) to sort. If sort-type is i, use student number to sort. If sort-type is n, use name to sort (or line from students list if equal). --useall all if all is 0 or empty, only students with a scaned answer sheet appears in the output. If all is 1, all students in the students list appears in the output. MODULES
OpenDocument With "--module ods", an OpenDocument (for use by OpenOffice or LibreOffice for example) is produced. The following options are recognized: nom gives a name for the exam, to be written on the beginning of the sheet. code gives an short name for the exam, to be used as a tab name. columns sets the list of columns that are to be added to each student. Default value is student.key,student.name. stats with a true value, a table will be added with statistics for all questions. statsindic with a true value, a table will be added with statistics for all indicative questions. CSV With "--module CSV", a CSV file is produced. The following options are recognized: columns sets the list of columns that are to be added to each student. Default value is student.copy,student.key,student.name. decimal sets the decimal point (default is a dot). encodage sets the encoding to use for output (default is utf-8). separateur sets the character used between colums (default is a comma). entoure sets the character used to enclose strings (default is a double quote). ticked if not empty, additional columns (with "CHECKED:" prefix) are included to give all the checked boxes on all the sheets. This can be used by an external program to retrieve all the checked boxes when marks are not sufficient. Use value 01 to get 0;0;1;1;0 if boxes 3 and 4 are checked, and value AB to get CD if boxes 3 and 4 are checked. List With "--module List", a PDF file is produced with a list of marks for all students. The following options are recognized: pagesize The page size. Default value is a4. nom gives the name of the exam, to be printed at the top of the page. ncols The number of columns. Default value is 2. decimal sets the decimal point (default is a dot). AUTHORS
Alexis Bienvenue <paamc@passoire.fr> Main author Jean Berard Translation from French Georges Khaznadar Translation from French COPYRIGHT
Copyright (C) 2008-2012 Alexis Bienvenue This document can be used according to the terms of the GNU General Public License, version 2 or later. Auto Multiple Choice 1.1.1 06/19/2012 AMC-EXPORT(1)
All times are GMT -4. The time now is 09:36 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy