Train and test-sets are not compatible error in weka - weka

Question

I'm trying to do text-classification in Weka, but I'm having a lot of problem getting the test-set to work. Here's my training set (it's short as I'm just starting to learn weka!):
#relation sentiment
#attribute phrase string
#attribute value {pos, neg}
#data
'That was really unlucky', neg
'The car crashed horribly', neg
'The culpirit got away',neg
'Fortunally everyone made it out', pos
'She was glad noone was hurt',pos
'And the sun was at least shining',pos
I then use StringToWordVector on the set, and then apply the NumericToBinary. This is the end result of the training set:
#relation 'sentiment-weka.filters.unsupervised.attribute.StringToWordVector-R1-W1000-prune-rate-1.0-N0-stemmerweka.core.stemmers.NullStemmer-M1-tokenizerweka.core.tokenizers.WordTokenizer -delimiters \" \\r\\n\\t.,;:\\\'\\\"()?!\"-weka.filters.unsupervised.attribute.NumericToBinary'
#attribute value {pos,neg}
#attribute And_binarized {0,1}
#attribute Fortunally_binarized {0,1}
#attribute She_binarized {0,1}
#attribute at_binarized {0,1}
#attribute everyone_binarized {0,1}
#attribute glad_binarized {0,1}
#attribute hurt_binarized {0,1}
#attribute it_binarized {0,1}
#attribute least_binarized {0,1}
#attribute made_binarized {0,1}
#attribute noone_binarized {0,1}
#attribute out_binarized {0,1}
#attribute shining_binarized {0,1}
#attribute sun_binarized {0,1}
#attribute the_binarized {0,1}
#attribute was_binarized {0,1}
#attribute That_binarized {0,1}
#attribute The_binarized {0,1}
#attribute away_binarized {0,1}
#attribute car_binarized {0,1}
#attribute crashed_binarized {0,1}
#attribute culpirit_binarized {0,1}
#attribute got_binarized {0,1}
#attribute horribly_binarized {0,1}
#attribute really_binarized {0,1}
#attribute unlucky numeric
#data
{0 neg,16 1,17 1,25 1,26 1}
{0 neg,18 1,20 1,21 1,24 1}
{0 neg,18 1,19 1,22 1,23 1}
{2 1,5 1,8 1,10 1,12 1}
{3 1,6 1,7 1,11 1,16 1}
{1 1,4 1,9 1,13 1,14 1,15 1,16 1}
I now start working on the testing set, which is:
#relation sentiment
#attribute phrase string
#data
'That was really unlucky'
'The car crashed horribly'
'The culpirit got away'
My hope is that weka can classify this text as 'neg'. To make them compatible I use the same filters as I did on the training set (StringToWordVector and NumericToBinary). This is the end result of the test-set:
#relation 'sentiment-weka.filters.unsupervised.attribute.StringToWordVector-R1-W1000-prune-rate-1.0-N0-stemmerweka.core.stemmers.NullStemmer-M1-O-tokenizerweka.core.tokenizers.WordTokenizer -delimiters \" \\r\\n\\t.,;:\\\'\\\"()?!\"-weka.filters.unsupervised.attribute.NumericToBinary'
#attribute That_binarized {0,1}
#attribute The_binarized {0,1}
#attribute away_binarized {0,1}
#attribute car_binarized {0,1}
#attribute crashed_binarized {0,1}
#attribute culpirit_binarized {0,1}
#attribute got_binarized {0,1}
#attribute horribly_binarized {0,1}
#attribute really_binarized {0,1}
#attribute unlucky_binarized {0,1}
#attribute was numeric
#data
{0 1,8 1,9 1,10 1}
{1 1,3 1,4 1,7 1}
{1 1,2 1,5 1,6 1}
However, it gives me the error that the training set and the test-set are not compatible, and I can't really figure out why. This intuitively seems like something that weka should understand.
Thanks for any help!

Answer 1

Your training and testing test should have same header. Right now they are different.
Read following link for an example for text classification.. This is another link which shows other ways to solve this problem.