It is known by the classical book "Einstein Manifolds" (Besse, 1984) that quasi-Einstein manifolds correspond to a base of a warped product Einstein metric. Another interesting motivation to investigate quasi-Einstein manifolds derives from the study of diffusion operators by Bakry and Emery (1985), which is linked to the theories of smooth metric measure space, static spaces and Ricci solitons. In this talk, we will show that a 3-dimensional simply connected compact quasi-Einstein manifold with boundary and constant scalar curvature must be isometric to either the standard hemisphere S^3_{+}, or the cylinder R x S^2 with product metric. For dimension n=4, we will show that a 4-dimensional simply connected compact quasi-Einstein manifold with boundary and constant scalar curvature is isometric to either the standard hemisphere S^4_+, or the cylinder I x S^3 with product metric, or the product space S^2_+ x S^2 with the product metric. This is a joint work with D. Zhou and J. Costa.