They may provide less comfort. Due to a decreased sidewall, 20 inch Rims can increase discomfort when going over potholes, speed bumps, gravel roads, and other road hazards.
Another Disadvantage, without a doubt, is handling. Remember, the giant wheel, and the smaller tire sidewall will affect cornering and performance. If you're into off-roading, 20-inch tires might not be the best choice.