Adding Fields of growing fields Unix - regex

Question

I have a grade book file that looks like
StudentID:LastName:FirstName:hw01:quiz01:exam01:proj01:quiz02:
0123:Smith:Jon:100:80:80:100:90:
0987:Williams:Pat:20:30:35:46:50:
0654:Bar:Foo:100:100:100:100:100:
I need to add all the hws/quizes/exams/projects for each student and append the total to the end of the corresponding line
An example output file could be
StudentID:LastName:FirstName:hw01:quiz01:exam01:proj01:quiz02:hT:qT:eT:pT
0123:Smith:Jon:100:80:80:100:90:100:170:80:100:
0987:Williams:Pat:20:30:35:46:50:20:80:35:46:
0654:Bar:Foo:100:100:100:100:100:100:200:100:100:
The output file doesn't have to be the same file, but keep in mind that the order of the grades in the header line (1st line) could be anything. So the order of the assignments could be in any order.
I'm assuming I must use grep to search the file for all fields containing "hw"/"quiz"/"exam"/"proj" and get the corresponding field. Then go through each line and add the totals for hw/quiz/exam/proj individually.
Maybe it's easier with awk?

Answer 1

$ cat tst.awk
BEGIN { FS=OFS=":" }
NR==1 {
for (i=4;i<NF;i++) {
name = substr($i,1,1) "T"
nr2name[i] = name
if (!seen[name]++) {
names[++numNames] = name
}
}
printf "%s", $0
for (nameNr=1; nameNr<=numNames; nameNr++) {
printf "%s%s", names[nameNr], OFS
}
print ""
next
}
{
delete tot
for (i=4;i<NF;i++) {
name = nr2name[i]
tot[name] += $i
}
printf "%s", $0
for (nameNr=1; nameNr<=numNames; nameNr++) {
printf "%s%s", tot[names[nameNr]], OFS
}
print ""
}
.
$ awk -f tst.awk file
StudentID:LastName:FirstName:hw01:quiz01:exam01:proj01:quiz02:hT:qT:eT:pT:
0123:Smith:Jon:100:80:80:100:90:100:170:80:100:
0987:Williams:Pat:20:30:35:46:50:20:80:35:46:
0654:Bar:Foo:100:100:100:100:100:100:200:100:100:

Answer 2

This seems to do the job; it is intricate, though:
script.awk
BEGIN { FS = ":"; OFS = FS }
NR == 1 {
for (i = 4; i < NF; i++)
{
c = substr($i, 1, 1)
if (!(c in columns)) order[n++] = c
columns[c]++
letter[i] = c
}
nf = NF
for (i = 0; i < n; i++)
$(i+nf) = order[i] "T"
print $0 OFS
next
}
{
for (c in columns) total[c] = 0
for (i = 4; i < NF; i++)
{
c = letter[i]
total[c] += $i
}
nf = NF
for (i = 0; i < n; i++)
{
c = order[i]
$(i+nf) = total[c]
}
print $0 OFS
}
Explanation:
BEGIN:
Set the input and output field separators.
NR == 1:
Loop over the fields after the student ID and name fields.
Extract the first letter.
If the letter has not been seen before, note it in the order and increment the number of totals (n).
Increment the number of times the letter has been seen.
Record which letter goes with the current column.
Add the new columns after the existing columns in sequence.
Print the line plus a trailing output field separator (aka OFS or :).
Note that $NF is empty because of the trailing : in the data, hence (unusually for awk scripts), i < NF rather than i <= NF.
Each other line:
Reset the totals for each letter.
For each of the scoring fields, find the letter that the column belongs to (letter[i]).
Add the column to the total for the letter.
For each of the extra fields in order, set the value of the appropriate extra field to the total for that letter.
Print the record plus an extra colon (aka OFS).
data
StudentID:LastName:FirstName:hw01:quiz01:exam01:proj01:quiz02:
0123:Smith:Jon:100:80:80:100:90:
0987:Williams:Pat:20:30:35:46:50:
0654:Bar:Foo:100:100:100:100:100:
Sample output
$ awk -f script.awk data
StudentID:LastName:FirstName:hw01:quiz01:exam01:proj01:quiz02:hT:qT:eT:pT:
0123:Smith:Jon:100:80:80:100:90:100:170:80:100:
0987:Williams:Pat:20:30:35:46:50:20:80:35:46:
0654:Bar:Foo:100:100:100:100:100:100:200:100:100:
$
The only difference between this and the sample output in the question is a trailing colon on the title line, for consistency with the data lines (and input).

Answer 3

with few adaptation
- Order of Total is not the same (dynamic)
- Name of total is using full radical names without the last 2 digit
- Use a parameter to define first field with data to count (4th here with -v 'St=4')
awk -v 'St=4' '
BEGIN{FS=OFS=":"}
NR==1 {
printf "%s",$0
for(i=St;i<=(nf=NF-1);i++){
tn=$i;sub(/..$/,"T",tn)
T[tn]=0;TN[i]=tn
}
Sep=""
for(t in T){
printf "%s%s",Sep,t;Sep=OFS
}
print Sep
next
}
{
for(i=St;i<=nf;i++){
T[TN[i]]+=$i
}
for(i=1;i<=nf;i++)printf "%s%s",$i,OFS
Sep=""
for(t in T){
printf "%s%s",Sep,T[t]
T[t]=0;Sep=OFS
}
print Sep
}' YourFile
StudentID:LastName:FirstName:hw01:quiz01:exam01:proj01:quiz02:examT:quizT:hwT:projT:
0123:Smith:Jon:100:80:80:100:90:80:170:100:100:
0987:Williams:Pat:20:30:35:46:50:35:80:20:46:
0654:Bar:Foo:100:100:100:100:100:100:200:100:100: