Originally Posted by JaneD0e
I am trying to scrape a website that has a large amount of .mp4 videos on it. I have tried a few tools like youtube-dl and you-get with out any luck.
Looking to see if there is any one with some python or programming skills that might be able to help,
Login to web site (Using cookies.txt) or Username /Password
Take a list of base urls from a text file
Spider that web page for any links that start with /player.aspx
Download any urls that have "var VideoSource_HD ="
Take the next base url repeat until all pages have been scraped.
Happy to share all the content scraped.
Looking to see if there is any one with some python or programming skills that might be able to help,
Login to web site (Using cookies.txt) or Username /Password
Take a list of base urls from a text file
Spider that web page for any links that start with /player.aspx
Download any urls that have "var VideoSource_HD ="
Take the next base url repeat until all pages have been scraped.
Happy to share all the content scraped.
I tried something, but didn't test it.
import bs4
import requests
res = requests.get('URL')
#res.text can give you the whole page source
soup = bs4.BeautifulSoup(res.text, 'lxml')
for i in soup.select('var VideoSource_HD ='):
print(i.text)
