Hello, as per your query you have been given a Belkin Wireless router but without a CD or Ethernet cable. The Comcast being 2 stories below the router is a bit confusing but I will try to answer your query keeping in mind the issue of CD and ethernet cable.
You can easily setup your router by bypassing your modem, Comcast in this case but you will need an Ethernet cable for sure. It is cheap and easily available so it's better to get one and then proceed to the settings. Follow the instructions carefully:
Step 1. Connect your computer to the router using an ethernet cable. First, connect the internet cable to the internet port of your router bypassing the modem also known as WAN port.
Now using an ethernet cable connect your computer to the router. One end of the cable goes inside one of the LAN ports on your router and the other end goes inside the Ethernet port of your computer.
Wait for few minutes to let the connection between the two get established.
Step 3. Now your router will detect that it is being configured for the first time and redirect you to "Let us connect to internet" page, here click on "Detect my connection".
Step 4. Once the connection has been tested and established between your router and internet, you will need to modify your network name and password. You can find the default wireless setup at the bottom of your router. Once the settings have been changed save it.
Step 5. Your router will now verify the network name and password after that is complete you need to fill in a few details on the screen and complete the registration process.
Step 6. Now go to your main router dashboard and change your security settings. Make your encryption as WPA2 and choose a secure password which is just below the encryption type. Also, change your IP settings to dynamic IP so that your network continuously provides IP address. Save these settings and close the window.
Hopefully, these steps helped you resolve the problem. In case you have any difficulty in understanding these steps kindly revert back to your query and I will be glad to help you out.