Just an Idea I had what about recruiters?
Like putting one of your guys in a position to recruit other members to the town so and based upon their skill in that particular area would determine how good the recruits are. For example you have a scientist recruiter and well he's never touched the research bench so he would recruit lower par recruits as opposed to a guy who spent the entirety of the town building process would recruit better prospects.
Same goes for soldiering, mining and just everything, however these recruits that they bring in you can control so they would be like townsmen or immigrants making thief stake in your town.
So they would have the option of leaving if they felt like they weren't being treated right or other factors that would possibly be thrown in.