The "Rocket Man" and "Candle in the Wind" singer and his partner David Furnish confirmed the news in a short statement on John's official website, which also provided a link to an article in Hello! magazine.
"Both of us have longed to have children, but the reality that we now have two sons is almost unbelievable," said the couple, who entered a civil partnership in 2005.
"The birth of our second son completes our family in a most precious and perfect way," they told Hello!.
John, 65, and Furnish, 50, are already parents to Zachary, who is two. Elijah was born in Los Angeles on January 11.
"I know when he goes to school there's going to be an awful lot of pressure, and I know he's going to have people saying, 'You don't have a mummy,'" John said of his decision to have another baby.
"It's going to happen. We talked about it before we had him. I want someone to be at his side and back him up. We shall see."
(This story has been corrected to change magazine to Hello! from People in paragraph two)
(Reporting by Mike Collett-White, editing by Paul Casciato)
