UNIX for Beginners Questions & Answers

Ubuntum, Bash version: 4.3.46
Bash, Perl

The position of XXX columns change in each file

Input file

aaa bbb XXX ddd XXX XXX   <-- Title
123 afa 133 2e2 dqd 24f
134 feg 566 5tf erg fe4
546 rgr 135 g5r hyt grt


Output file

aaa bbb ddd   <-- Title
123 afa 2e2
134 feg 5tf
546 rgr g5r

#!/usr/bin/env bash

# @(#) s1       Demonstrate elimination of some fields, csvtool.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
em() { pe "$*" >&2 ; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C csvtool pass-fail

FILE=${1-data1}
E=expected-output.txt

pl " Input data file $FILE:"
cat $FILE

pl " Expected output:"
cat $E

# Extract names, remove XXX, separate remainder with commas.
keeping=$( head -1 $FILE | sed 's/ XXX//g ; s/ /,/g' )
pl " Keeping these columns:" $keeping

pl " Results:"
csvtool -t " " -u " " namedcol $keeping  $FILE |
tee f1

pl " Verify results if possible:"
C=$HOME/bin/pass-fail
[ -f $C ] && $C || ( pe; pe " Results cannot be verified." ) >&2

exit 0

$ ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.6 (jessie) 
bash GNU bash 4.3.30
csvtool - ( /usr/bin/csvtool, 2014-08-06 )
pass-fail (local) 1.9

-----
 Input data file data1:
aaa bbb XXX ddd XXX XXX
123 afa 133 2e2 dqd 24f
134 feg 566 5tf erg fe4
546 rgr 135 g5r hyt grt

-----
 Expected output:
aaa bbb ddd
123 afa 2e2
134 feg 5tf
546 rgr g5r

-----
 Keeping these columns: aaa,bbb,ddd

-----
 Results:
aaa bbb ddd
123 afa 2e2
134 feg 5tf
546 rgr g5r

-----
 Verify results if possible:

-----
 Comparison of 4 created lines with 4 lines of desired results:
 Succeeded -- files (computed) f1 and (standard) expected-output.txt have same content.

csvtool tool for performing manipulations on CSV files from sh... (man)
Path    : /usr/bin/csvtool
Version : - ( /usr/bin/csvtool, 2014-08-06 )
Type    : ELF 64-bit LSB executable, x86-64, version 1 (SYSV ...)
Help    : probably available with --help
Home    : https://github.com/Chris00/ocaml-csv

awk -vRM="XXX" 'NR == 1 {for (i=1; i<=NF; i++) if ($i == RM) DL[i]} {for (i in DL) $i = ""; gsub (FS FS, FS)} 1' file
aaa bbb ddd
123 afa 2e2
134 feg 5tf
546 rgr g5r