Here is an interesting passage from Steven Pinker’s The Blank Slate:
In any case, there is now ample evidence that intelligence is a stable property of an individual, that it can be linked to features in the brain (including overall size, amount of gray matter in the frontal lobes, speed of neural conduction, and metabolism of cerebral glucose), that it is partly heritable among individuals, and that it predicts some of the variation in life outcomes such as income and social status. (p.150 paperback)
Conceptually, we can separate the question “Is there a physical/biological/genetic basis to intelligence” from the question “What moral implications does that have for society?” Still, it seems clear to me that there are moral implications that arise from the diversity in human intelligence. They are, however, connected to tricky moral problems, like what it means to ‘deserve’ one’s level of success or failure.