||Accurate estimation of Energy Expenditure (EE) and cardiorespiratory fitness (CRF) is a key element in determining the causal relation between aspects of human behavior related to physical activity and health. In this paper we estimate CRF without requiring laboratory protocols and personalize energy expenditure (EE) estimation models that rely on heart rate data, using CRF. CRF influences the relation between heart rate and EE. Thus, EE estimation based on heart rate typically requires individual calibration. Our modeling technique relies on a hierarchical approach using Bayesian modeling for both CRF and EE estimation models. By including CRF level in a hierarchical Bayesian model, we avoid the need for individual calibration or explicit heart rate normalization since CRF accounts for the different relation between heart rate and EE in different individuals. Our method first estimates CRF level from heart rate during low intensity activities of daily living, showing that CRF can be determined without specific protocols. Reference $VO_2$max and EE were collected on a sample of $32$ participants with varying CRF level. CRF estimation error could be reduced up to $27.0%$ compared to other models. Secondly, we show that including CRF as a group level predictor in a hierarchical model for EE estimation accounts for the relation between CRF, heart rate and EE. Thus, reducing EE estimation error by $18.2%$ on average. Our results provide evidence that hierarchical modeling is a promising technique for generalized CRF estimation from activities of daily living and personalized EE estimation.