Issues with links to posts/categories/archive

Hi,
I’m new to wordpress, and I’ve searched the forum to find similar problems I’m facing but none found.

Basically I’ve installed WordPress 2.0.1 to one of our web server (internal), got everything work out, admin works, people can login and post. But viewing a post or viewing an archive or viewing a category gives a broken link.
the link that it goes to is:
for posts – http://webserver/wordpress/index.php/2006/02/17/delphin-lend-lease-request-proposal-for-major-community-development/
for archive – http://webserver/wordpress/index.php/2006/02/
for category – http://webserver/wordpress/index.php/category/residential/

To me, the index.php in the middle does not look right at all.

Can anyone help me what it’s supposed to be, and how to fix it?