Can't recruit men in the north

[Kirky14-]

It is almost impossible to have a northern army, every village I go to either doesn't have the option to recruit or will only give me 1 or 2 peasants. The religion seems to have no effect either since whether I worship the old gods or faith of the seven doesn't make a difference. I'm not sure if this happens elsewhere in the map since I have not tried it.