The History of Search Engine Marketing (SEM)

August 3, 2007

Searching the Internet in the pre-World Wide Web days required downloading and installing programs which would then download and search directory listings from public FTP sites. It was all rather complex, and not for the non-geek. Which was fine since only geeks perused, or even knew of the existence of, the Internet in those days.

In the early 90’s, the World Wide Web was born and began to grow rapidly. Graphical Internet browsers were developed to make “surfing” the Web easy. But finding a webpage required that you knew its URL. Therefore, if you found a website you liked, having some method of keeping track of it was very helpful.

For this reason, browsers developed the bookmark, allowing you to save the location of a specific webpage, mapping the URL to a shortcut button within the browser. To revisit the webpage, you simply opened your bookmarks and clicked the appropriate shortcut link.

However, there still wasn’t any simple method of finding specific websites if you didn’t know the URL. In 1994, graduate students Jerry Yang and David Filo created a website called “Jerry’s Guide to the World Wide Web” in which they essentially published their bookmarks. Over time, it grew into a searchable directory.

Within a few months, the name was changed to “Yahoo!”. Yahoo! helped make the Internet more user friendly and updated search technology for the World Wide Web.

However, Yahoo! was just a directory. It wasn’t capable of searching anything that hadn’t been previously submitted to it. Also released in 1994 was what would become the first popular search engine as we understand them today. “WebCrawler” actively crawled the web, indexing websites automatically instead of waiting for somebody to manually submit them to a directory.

For the first time, Internet users could search a database containing virtually every website available on the World Wide Web. Internet users quickly realized the value of WebCrawler and it became one of the web’s most popular websites.

Copycats soon followed. Webmasters began to realize that getting listed in these new search engines was a way to get traffic with very little work. Thus was born a new science of marketing via search engines, or search engine marketing (SEM) as it is known today.

Early search engines were relatively simple, and easily manipulated. Search engine indexes quickly became overrun with websites utilizing “search engine spam.”
Internet users became increasingly frustrated with the search results being provided by search engines. Then in 2001, Google was discovered and quickly became the most popular search engine. The reason for Google’s popularity was its ability to provide highly relevant search results. Google had created a new way of indexing webpages that negated the spam tactics being used by many webmasters.

Although it wasn’t perfect, Google’s methodology was far better than what was currently available, and so many other search engines began to follow suit, changing their algorithms to provide more relevant results.

Google wasn’t infallible however, and webmasters quickly began discovering ways to trick Google as well.

This has resulted in a never-ending battle between search engines and webmasters. As the search engines change their algorithms, webmasters continually update their sites in attempts to gain or maintain search engine rankings.

Failing the ability to get ranked, webmasters found a new tool in 1998 called Goto.com. Goto.com was the first popular pay-per-click (PPC) service that allowed you get traffic by paying on a per-click basis. As PPC grew in popularity, paid ads began appearing on the popular search engines as “sponsored” results.

Internet marketing today has become a science, requiring highly educated and experienced Internet and search engine experts that continually monitor search traffic and keep up on the latest news. Companies desiring a strong Internet presence now have entire departments dedicated to Internet marketing. For smaller companies that don’t have the resources to fund an entire separate department for it, Internet marketing companies like OrangeSoda can provide those services at a much smaller cost.