Using 180 pb-1 of data collected with the CDF II detector in Run II of the Tevatron, we measure the first two moments of the invariant mass-squared distribution of the charm-hadron system in semileptonic B decays. We measure directly the first two moments of the D** component for a minimum lepton momentum in the B rest frame of 0.7 GeV/c to be (m2_D**) = (5.83 +/- 0.16_stat +/- 0.08_syst) GeV2, ((m2_D** - (m2_D**))2) = (1.30 +/- 0.69_stat +/- 0.20_syst) GeV4. After combining them with the known contributions from D and D* we find for the moments of the charm-hadron system (M2_chi(c)) - m 2/D = (0.459 +/- 0.037_stat +/- 0.065_syst) GeV2, ((M2_chi(c) - (M2_chi(c)))2) = 1.04 +/- 0.25_stat +/- 0.12_syst) GeV4, where m_D-bar is the spin-averaged D mass and the systematic error is dominated by the uncertainty in the branching ratios used in the combination of the D, D* and D** pieces. From the moments we determine the non-perturbative HQET parameters Delta and lambda_1 in the pole and 1S mass schemes.