[Release] Web Scraping Toolkit (Class Library)

**Jason** · 03-08-2011

Hey guys, spent a little time tonight knocking up a class library to include some of the major functions I like to use to make various web-scraping project. I only added really general functions to this library as you'll obviously need to make some custom shit depending on the information you need from each webpage, but these will sure make it easier.

Features in this Library:

How to use the class
First you need to reference the library (Project -> Add Reference -> Browse -> Locate Dll)

Next, you should import the namespace
[highlight=vb.net]
Imports WebClasses
[/highlight]

Now to create a new instance of the class:
[highlight=vb.net]
Dim wClass As New WebClass
[/highlight]

Now you call the classes methods from your instance of the class (I documented all the methods, so you'll be able to see their functionality when you use the library)

Anyway, I made this for myself as I do lots and LOTS of webscraping, and got sick of rewriting a lot of functions. But, I hope this assists others.

Virus Scans:

VirusTotal - Free Online Virus, Malware and URL Scanner

WebClasses.zip MD5:0c5ce2a77fa30574158dde39acb72fd3 - VirSCAN.org Scanners did not find malware!

Brought to you by MPGH Official Development and Design Team.

Coded exclusively by: Jason.

Enjoy niggaz

**Withoutwings** · 03-08-2011

Nice, as expected.

**Lyoto Machida** · 03-08-2011

Very nice, But i never understand things like this, PRO things

**Skinnlaw** · 03-08-2011

Nice like Jason always nice

**Blubb1337** · 03-08-2011

Credits for telling you how to do those sexy function previews

?

**Hassan** · 03-08-2011

Very Nice, but I expected more from this library.
+
Credits for telling you to include ParseAllImages and ParseAllLinks functions ? /fmm :P

Originally Posted by Blubb1337

Credits for telling you how to do those sexy function previews /mmz?

I told you how to do that, yet you never gave credits ? /fosho :P

**nathanael890** · 03-08-2011

Very nice GUI!
If you don't mind, can you share it to us? I'm collecting them

**Hassan** · 03-08-2011

Originally Posted by nathanael890

Very nice GUI!
If you don't mind, can you share it to us? I'm collecting them

It's a release. He already shared ?

**nathanael890** · 03-08-2011

Oops.. I thought it was just a project ^^

**Jason** · 03-08-2011

Originally Posted by Hassan

Very Nice, but I expected more from this library.
+
Credits for telling you to include ParseAllImages and ParseAllLinks functions ?

:P

I told you how to do that, yet you never gave credits ?

:P

I couldn't think of much else to include

**Hassan** · 03-08-2011

Originally Posted by Jason

I couldn't think of much else to include

You dissapointed me son. There are tons and tons of thing you can include (not mentioning the one's of msn). I'll make parsing and scrapping library soon.

**Jason** · 03-09-2011

Originally Posted by Hassan

You dissapointed me son. There are tons and tons of thing you can include (not mentioning the one's of msn). I'll make parsing and scrapping library soon.

The one's you said in MSN were deemed useless by me. Like I said in MSN, I've never EVER even formed a half thought of using one of those functions. So why spend time writing the function? This library allows me to scrape sites with steez and ease. If you can't do it...too bad for you. Although I do remember when I first wanted to scrape sites you told me to use a streamreader and go through the page line-by-line

.

Originally Posted by "Hassan

I'll make parsing and scrapping library soon.

No, you won't. You'll say you will, fuck around for a while then never release anything. What happened to your 5 "epic releases that will pwn this section" and the Uber Advanced WebBrowser tutorial with GeckoFX?