The allegations were made recently by the WikiLeaks site.
Abe told Biden that if the allegations were true, "it could rattle the foundation of trust between the two allied countries and I would have no choice but to express my deepest concern."
He also requested that the Japanese government be informed if a U.S. investigation into the matter reached any conclusion.
In response, Biden explained the details of the alleged telephone intercepts and apologized, saying he regretted that the issue was a cause for concern for the prime minister and the Japanese government.
The vice president also reconfirmed that the United States was committed to the presidential directive issued by President Barack Obama last year to focus its intelligence collection on U.S. national security interests. He assured Abe that the U.S. government was not engaged in any activity that would disrupt trust between the two countries.
The telephone conversation was held at the request of the U.S. government.
At a press conference Wednesday, Chief Cabinet Secretary Yoshihide Suga said, "I would like to refrain from responding [to questions regarding alleged telephone intercepts in the past]."