Your right that really the only people who believed it could be done from the start was von Braun and Korolev. In fact, when Yuri Gagarin because the first man to fly into space on Vostok 1 there was a lot of concern that we had lost the "race" and they had to decide what to set the national goal too. A lot of people in NASA wanted to build an oribital lab (space station) instead of going to the moon. But, it was decided that if the US and USSR got into a race over heavy lifting (all a space station would show) then we would lose until probably 1966 because we were behind with rocket designs. A moon shot was a better idea because not only would a strong rocket be needed (which the Russian's were ahead with) but special technology which hadn't been invented yet (which were were leading with). So, it was thought that we had a better chance to win and thats why Kennedy promised the moon in that speech.
One interesting thing though, it may not have been nearly as big of a deal had Werner von Braun not been as causious. In March of 1961, (1 month before Gagarin's flight), NASA was planning a manned Mercury-Redstone flight which would have out the first man in space. When the decision time came, von Braun wasn't completely comfortable with the safety of the rocket and capsule and wanted one more unmanned test, so an unmanned flight was launched in March (the manned mission bumped to May). The unmanned flight went perfectly. In April and Russian's launched Vostok 1 (manned orbital flight) and the US found themselves behind. In May, "Freedom 7" flew with Alan Shepard onboard putting the first American in space on a balistic suborbital flight. Had NASA manned the March flight and put the first man in space, Kennedy may never had made that speech and a race to the moon may not have happened (at least not by the end of the 1960s).
Richie. I think you have to appreciate that the crews on those early manned missions were little more than passengers. The first mammals in space had been chimpanzees & dogs that had no control over the space vehicles at all. The computers available at the time were simply too bulky & heavy to consider carrying in the spacecraft & would have remained on the ground where the major part of the mission was controlled from.
Your right, the computers were too heavy. But, the Mercury astronauts did have some control of the capsule. In fact, astronaut Scott Carpenter took control of his "Aurora 7" capsule during reenrty because he believed he was offcourse. As it turned out, his instruments were wrong so he didn't need to take manual control-but he did successifully land his capsule manually (although offcourse by about 200 miles). The Russian Vostok and Voskhod were completely automated though-the cosmonaut not only didn't have control normally, but if he needed to take control he had to open a sealed envelope and punch in a unlocking code. But, this was only allowed under extreme circumstances and if the cosmonaut had taken control without need, punishment was said to be severe. But, that was never needed.