you're basically screwed. you can't go to the labor board and say, "hey, the guy was paying me under teh table......bla bla bla...."
since that would make your intentions illegal. it seems he's labeled you 'contract worker' which you not only need to pay state and federal taxes from but all
of your social security taxes as well. if you are a direct employee of a company, they pay 1/2 your social security taxes. since you are a contract worker, you must pay all your social security taxes. did you make any tips throughout the year? you must claim those as well. if he's done the same thing to your co workers, tell them this and see if they now think you're being to harsh. you've all been screwed