• FauxLiving@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      2
      ·
      4 days ago

      There’s a pretty significant difference in request rate. A tool trying to search and summarize will hit a search engine once, and each website maybe 5 times (if every search engine link points to the site).

      A bot trying to scrape content from a website can generate thousands or tens of thousands of requests per second.