There is little reason for concern about the second group since the United States has asserted the authority to spy on foreigners overseas and there is no statute or Constitutional provision that would prohibit the US from doing so. As far as the collection of data regarding phone numbers and the duration of phone calls conducted within the United States, since 1979 it has been clear that such collections are Constitutional.
In 1979 the Supreme Court decided the case, Smith the Maryland, which held that there was no reasonable Fourth Amendment expectation of privacy regarding the phone numbers given by a person to a phone company. In that case, the police installed a "pen register" at a telephone company without a warrant. The pen register recorded all of the phone numbers that were dialed from the defendant's home telephone. The police use those numbers to develop evidence against the defendant. At trial the defendant sought to have that evidence excluded on the grounds that it was only obtained through a warrantless search of his phone records. The lower courts and ultimately the Supreme Court ruled that there was no Fourth Amendment violation. Specifically, the Supreme Court stated:
(a) Application of the Fourth Amendment depends on whether the person invoking its protection can claim a "legitimate expectation of privacy" that has been invaded by government action. This inquiry normally embraces two questions: first, whether the individual has exhibited an actual (subjective) expectation of privacy; and second, whether his expectation is one that society is prepared to recognize as "reasonable." Katz v. United States, 389 U. S. 347. Pp. 442 U. S. 739-741.
(b) Petitioner in all probability entertained no actual expectation of privacy in the phone numbers he dialed, and even if he did, his expectation was not "legitimate." First, it is doubtful that telephone users in general have any expectation of privacy regarding the numbers they dial, since they typically know that they must convey phone numbers to the telephone company and that the company has facilities for recording this information and does, in fact, record it for various legitimate business purposes. And petitioner did not demonstrate an expectation of privacy merely by using his home phone, rather than some other phone, since his conduct, although perhaps calculated to keep the contents of his conversation private, was not calculated to preserve the privacy of the number he dialed. Second, even if petitioner did harbor some subjective expectation of privacy, this expectation was not one that society is prepared to recognize as "reasonable." When petitioner voluntarily conveyed numerical information to the phone company and "exposed" that information to its equipment in the normal course of business, he assumed the risk that the company would reveal the information to the police, cf. United States v. Miller, 425 U. S. 435. Pp. 442 U. S. 741-746.
In this case the NSA is merely collecting the phone numbers involved in phone calls and the duration of those calls. It does not, as in the Smith case, know the identity of the persons who use the phone numbers. Also, unlike in Smith, it obtained a warrant before seeking the information.
These numbers are used for one purpose. If the NSA has the phone number used by a suspected terrorist it can try to determine whether or not that phone number appears in the database. If the phone number does, it will then track which numbers that phone number is connected to. At that point, assuming it has a reasonable suspicion, it can go to the Foreign Intelligence Surveillance Court and seek a warrant to put a wiretap on the phone numbers. If the court agrees that there is a reasonable suspicion that this person is a terrorist or affiliated with the terrorist and that the phone number is one reasonably believed to have been used by that person then the court may issue a warrant after which the phone can be wiretapped.
I would be concerned if this program in any way resembled the Bush warrantless wiretapping program which was discontinued after significant outrage. The Bush warrantless wiretap program, which they referred to as the "terrorist surveillance program" involved the collection of the content of telephone calls, emails, and other communications. The Obama program does not. It only involves phone numbers and the duration of phone calls. There is no recording of telephone calls. If, based on a review of those phone numbers the authorities wish to obtain the content of future communications between people associated with those numbers, they have to obtain a separate individualized warrant similar to that required in any criminal case.
Additionally, the Bush program was done without any warrant or legal authority. The program was not authorized under Foreign Intelligence Surveillance Act and consequently no warrant was sought from a court. Instead Bush asserted it could be done through an Executive Order issued under the authority of the AUMF (Authorization For Use Of Military Force) and without court review.
If it turns out that the Obama administration has or in the future develops a program different from the one that is currently outlined I will have a problem with it. But until then I see this as a minimal intrusion.
One final observation. Most people are well aware that there are reasonable criticisms of the operations of the Foreign Intelligence Surveillance Court. It has been viewed by many as little more than a rubber stamp for the administration seeking to obtain the patina of court oversight over its intelligence gathering activities. I sympathize with those concerns. However, there are reasons why this court exists and operates in the way it does. The information provided to the FISA Court is generally classified and therefore must be presented to judges who have the appropriate clearances. Second, decisions often have to be made very fast, which requires the availability of judges who can make decisions any day and at any time of day. Finally these are specialized areas of the law that require judges with specialized knowledge. It is almost inevitable that any court designed to carry out the unique responsibilities of a FISA court will also have the limitations that we see in the FISA court.