• Brewchin@lemmy.world
    link
    fedilink
    English
    arrow-up
    71
    ·
    3 months ago

    Parts of the Internet now only searchable on specific sites now? What next - charging a monthly subscription to use Google?

    This needs to be regulated before the Internet becomes like streaming TV.

    • tal@lemmy.today
      link
      fedilink
      English
      arrow-up
      28
      arrow-down
      3
      ·
      edit-2
      3 months ago

      Robots.txt has been around for a long time, and all the major search engines will honor it. Not having a full index of the Web is the norm.

      That isn’t to say that the practice of signing agreements isn’t potentially a concern. Not sure that I like the idea of search engines paying sites money to degrade search results of competitors.