I cannot speak on other states or communities…….but in my area you may not build more than one home on your agricultural land, and you must own at least 20 acres to build any home on your land at all.
This means that a family farm cannot have more than one residence on the land, therefore if the owner would like to build a second residence for there adult children, or elderly parent to live on the property with them, they are not allowed to do this legally. This makes it difficult for family farms to pass the land and trade on to the next generation.
No County, or state should be able to do this. If you want to farm on 5 acres or 10 or 100 acres you should be able to build a residence on that land, and no one should be able to stop you, you should also be able to build a secondary house on your farm or ranch as this familial support is what keeps small ranches and farms going for generations.