Hey there, for tax inefficiency, you have to get your minister of coin to hire people that "intimidate even him" that will turn tax inefficiency around for you. As for the taxes, this also relates to the castles/villages associated with the town, so the town in question is only as rich as the villages & castles surrounding it. Things like sieges & pillages of castles & villages will hamper your towns taxes. Maybe place patrols around the closest villages & castles (even if they don't belong to you, or even to your faction in the case of castles!)
Hope this helps.
Hope this helps.