There are really only two usable search engines actually indexing the entire Internet: Google and Bing. Yandex also does but I’ve never seen it recommended for anything other than Russian language content (the company itself seems to be falling down a mineshaft at the moment). Baidu also does some although every Chinese exchange student I talked to about it (admittedly not many) advised only using it when Google is blocked. Every other engine is just wrapping Google or Bing (yes that includes Yahoo and DDG)
This is the kind of ugly truth of the search engine business. It’s a duopoly at least in part because the indicies are expensive to scrape, build, and run. You need to continuously run a large number of servers loading web pages and often running scripts. You need to be large enough to negotiate with content providers not to block you. Keep in mind paying them may bankrupt you as your margins will be thin. Google has a huge advantage here they own a good chunk of the online advertising industry and can afford to throw money around in a way a search only company wouldn’t be able to (this is why the European and Canadian link tax schemes ironically cement the existing monopolies). You need to continuously run large linear aglebra transforms on the results (PageRank is expensive). You need to store all your indicies on large expensive servers with a lot of memory as hitting disk may take too long. Results need to be fast and you will make next to nothing on each search.
There are really only two usable search engines actually indexing the entire Internet: Google and Bing. Yandex also does but I’ve never seen it recommended for anything other than Russian language content (the company itself seems to be falling down a mineshaft at the moment). Baidu also does some although every Chinese exchange student I talked to about it (admittedly not many) advised only using it when Google is blocked. Every other engine is just wrapping Google or Bing (yes that includes Yahoo and DDG)
This is the kind of ugly truth of the search engine business. It’s a duopoly at least in part because the indicies are expensive to scrape, build, and run. You need to continuously run a large number of servers loading web pages and often running scripts. You need to be large enough to negotiate with content providers not to block you. Keep in mind paying them may bankrupt you as your margins will be thin. Google has a huge advantage here they own a good chunk of the online advertising industry and can afford to throw money around in a way a search only company wouldn’t be able to (this is why the European and Canadian link tax schemes ironically cement the existing monopolies). You need to continuously run large linear aglebra transforms on the results (PageRank is expensive). You need to store all your indicies on large expensive servers with a lot of memory as hitting disk may take too long. Results need to be fast and you will make next to nothing on each search.