There is a fairly lengthy list of tasks when first starting a website. This is a compilation for my future sanity:

1. Choose a website name (do this first because you need it for all subsequent steps. Don’t continue until you’ve done this because it’s almost impossible to change later.
2. Register the URL. I recommend hosting with Dreamhost. They’re great value for money and the support is excellent. In general, it’s easier to register your URL through your hosting provider.
3. Submit the URL to all the search engines as soon as possible. The crawlers will take a while to getting around to your site.
4. Download the Aptana IDE. It’s a great, free editor.
5. Download and install Firefox. Install the Firebug plugin.
6. Make sure you have 301 redirects to ensure that Google sees your website as a single URL and not two different websites (one for www.domain.com and another for domain.com). If you’re using Dreamhost this is easily achieved by selecting your preferred URL format under the ‘Manage Domains’ section of the Dreamhost panel. There is no need for a .htaccess file if you’re using Phusion Passenger.
7. Sign up for Google Analytics, Google Webmaster and (if you plan on advertising) Google Adwords
8. When designing your initial web layout, leave space for ads if you plan to add them later.
9. Good link for embedding video
10. After a month or so, use Hubspot to check your search engine hygiene.

I’ll add more to this list over time.