Results 1 to 3 of 3
  1. #1
    Holmesdr's Avatar
    Join Date
    Nov 2020
    Gender
    male
    Location
    Amsterdam
    Posts
    6
    Reputation
    10
    Thanks
    0

    What are the rules associated with web scraping?

    I starting learned Beautiful Soup and want to test myself by doing some projects but I found not all websites allow web scraping and somethings about robots.txt. What are the legal things associated with it anyone advise me about what should I do or some projects?

  2. #2
    Holmesdr's Avatar
    Join Date
    Nov 2020
    Gender
    male
    Location
    Amsterdam
    Posts
    6
    Reputation
    10
    Thanks
    0
    I have found solutions for test my project https://mydataprovider.com/solutions/web-scraping/. It's provide any type of data extraction configuration the needs of me

  3. #3
    Matthew's Avatar
    Join Date
    Mar 2017
    Gender
    male
    Posts
    5,275
    Reputation
    1147
    Thanks
    1,138
    You could scrape all external links that the site gives you (does it redirect you to a third party, etc) or be boring and scrape the image links that are embedded in the site's code. I'm not to brushed up on web scraping

Similar Threads

  1. [Solved] What are the rules on video Tuts?
    By crietenz in forum Vindictus Help
    Replies: 5
    Last Post: 08-27-2011, 10:29 AM
  2. What are the chances of being caught with a wallhack?
    By thebeanie in forum Call of Duty Modern Warfare 2 Discussions
    Replies: 14
    Last Post: 12-03-2009, 07:56 PM
  3. When are the rules going to be enforced?
    By webbleking1.2 in forum General
    Replies: 1
    Last Post: 06-09-2008, 04:58 PM
  4. What are the types? help please
    By sidnietje in forum WarRock - International Hacks
    Replies: 0
    Last Post: 05-30-2007, 09:25 AM