Web scraper help

I am trying to scrape a website that has a large amount of .mp4 videos on it. I have tried a few tools like youtube-dl and you-get with out any luck.

Looking to see if there is any one with some python or programming skills that might be able to help,

Login to web site (Using cookies.txt) or Username /Password

Take a list of base urls from a text file

Spider that web page for any links that start with /player.aspx

Download any urls that have "var VideoSource_HD ="

Take the next base url repeat until all pages have been scraped.

Happy to share all the content scraped.

are you looking for someone who can do this work for you for free?

Someone who could guide me would be nice .

got most of this now built need help on regex

Any one know regex well ? can you then hit me up on a PM ?

re.findall("var VideoSource_HD =(.*?)\n", data)

Can any 1 do uber login bypass fetching script

Originally Posted by JaneD0e

I am trying to scrape a website that has a large amount of .mp4 videos on it. I have tried a few tools like youtube-dl and you-get with out any luck.

Looking to see if there is any one with some python or programming skills that might be able to help,

Login to web site (Using cookies.txt) or Username /Password

Take a list of base urls from a text file

Spider that web page for any links that start with /player.aspx

Download any urls that have "var VideoSource_HD ="

Take the next base url repeat until all pages have been scraped.

Happy to share all the content scraped.

I'm a noob too at web scraping.

I tried something, but didn't test it.

import bs4
import requests

res = requests.get('URL')
#res.text can give you the whole page source

soup = bs4.BeautifulSoup(res.text, 'lxml')

for i in soup.select('var VideoSource_HD ='):
print(i.text)

Thanks for all the help I was able to scrape the site in the end just took a lot of trial and error. Shows I need to start learning to code by doing a few online courses :-)

So will you be sharing all content scraped as promised now?

Web scraper help

Similar Threads

Tags for this Thread