Page 1 of 2 12 LastLast
Results 1 to 15 of 22
  1. #1
    Jason's Avatar
    Join Date
    Apr 2010
    Gender
    male
    Location
    /dev/null
    Posts
    5,704
    Reputation
    918
    Thanks
    7,682
    My Mood
    Mellow

    [Release] Web Scraping Toolkit (Class Library)

    Hey guys, spent a little time tonight knocking up a class library to include some of the major functions I like to use to make various web-scraping project. I only added really general functions to this library as you'll obviously need to make some custom shit depending on the information you need from each webpage, but these will sure make it easier.

    Features in this Library:



    How to use the class
    First you need to reference the library (Project -> Add Reference -> Browse -> Locate Dll)

    Next, you should import the namespace
    [highlight=vb.net]
    Imports WebClasses
    [/highlight]

    Now to create a new instance of the class:
    [highlight=vb.net]
    Dim wClass As New WebClass
    [/highlight]

    Now you call the classes methods from your instance of the class (I documented all the methods, so you'll be able to see their functionality when you use the library)

    Anyway, I made this for myself as I do lots and LOTS of webscraping, and got sick of rewriting a lot of functions. But, I hope this assists others.

    Virus Scans:

    VirusTotal - Free Online Virus, Malware and URL Scanner

    WebClasses.zip MD5:0c5ce2a77fa30574158dde39acb72fd3 - VirSCAN.org Scanners did not find malware!

    Brought to you by MPGH Official Development and Design Team.

    Coded exclusively by: Jason.

    Enjoy niggaz
    Last edited by Jason; 03-08-2011 at 12:02 PM.

    Quote Originally Posted by Jeremy S. Anderson
    There are only two things to come out of Berkley, Unix and LSD,
    and I don’t think this is a coincidence
    You can win the rat race,
    But you're still nothing but a fucking RAT.


    ++Latest Projects++
    [Open Source] Injection Library
    Simple PE Cipher
    FilthyHooker - Simple Hooking Class
    CLR Injector - Inject .NET dlls with ease
    Simple Injection - An in-depth look
    MPGH's .NET SDK
    eJect - Simple Injector
    Basic PE Explorer (BETA)

  2. The Following 12 Users Say Thank You to Jason For This Useful Post:

    Blubb1337 (03-08-2011),ebookmausopwerjugamau (01-15-2014),haris4you (05-15-2013),Hassan (03-08-2011),iNsidi0us (08-27-2011),kunal1212 (03-08-2011),[MPGH]master131 (03-08-2011),nathanael890 (03-08-2011),NextGen1 (04-14-2011),Rakuuza (05-06-2012),Void (03-08-2011),Withoutwings (03-08-2011)

  3. #2
    Withoutwings's Avatar
    Join Date
    Jan 2011
    Gender
    male
    Location
    The netherlands
    Posts
    118
    Reputation
    10
    Thanks
    15
    My Mood
    Angelic
    Nice, as expected.
    Last edited by Withoutwings; 03-08-2011 at 11:59 AM. Reason: fail spelling.

    Crushing all in it's path, 'cause I do not think we're invincible.

  4. #3
    Lyoto Machida's Avatar
    Join Date
    Jan 2011
    Gender
    male
    Location
    Far away with girls
    Posts
    3,734
    Reputation
    133
    Thanks
    1,621
    My Mood
    Aggressive
    Very nice, But i never understand things like this, PRO things

  5. #4
    Skinnlaw's Avatar
    Join Date
    Feb 2011
    Gender
    male
    Location
    Netherlands
    Posts
    204
    Reputation
    -1
    Thanks
    15
    My Mood
    Amazed
    Nice like Jason always nice

  6. #5
    Blubb1337's Avatar
    Join Date
    Sep 2009
    Gender
    male
    Location
    Germany
    Posts
    5,915
    Reputation
    161
    Thanks
    3,109
    Credits for telling you how to do those sexy function previews ?



  7. #6
    Hassan's Avatar
    Join Date
    May 2010
    Gender
    male
    Location
    System.Threading.Tasks
    Posts
    4,764
    Reputation
    495
    Thanks
    2,137
    My Mood
    Dead
    Very Nice, but I expected more from this library.
    +
    Credits for telling you to include ParseAllImages and ParseAllLinks functions ? /fmm :P

    Quote Originally Posted by Blubb1337
    Credits for telling you how to do those sexy function previews /mmz?
    I told you how to do that, yet you never gave credits ? /fosho :P

  8. #7
    nathanael890's Avatar
    Join Date
    Aug 2008
    Gender
    male
    Posts
    158
    Reputation
    13
    Thanks
    46
    My Mood
    Bored
    Very nice GUI!
    If you don't mind, can you share it to us? I'm collecting them


    Status: Unknown

  9. #8
    Hassan's Avatar
    Join Date
    May 2010
    Gender
    male
    Location
    System.Threading.Tasks
    Posts
    4,764
    Reputation
    495
    Thanks
    2,137
    My Mood
    Dead
    Quote Originally Posted by nathanael890 View Post
    Very nice GUI!
    If you don't mind, can you share it to us? I'm collecting them
    It's a release. He already shared ?

  10. #9
    nathanael890's Avatar
    Join Date
    Aug 2008
    Gender
    male
    Posts
    158
    Reputation
    13
    Thanks
    46
    My Mood
    Bored
    Oops.. I thought it was just a project ^^


    Status: Unknown

  11. #10
    Jason's Avatar
    Join Date
    Apr 2010
    Gender
    male
    Location
    /dev/null
    Posts
    5,704
    Reputation
    918
    Thanks
    7,682
    My Mood
    Mellow
    Quote Originally Posted by Hassan View Post
    Very Nice, but I expected more from this library.
    +
    Credits for telling you to include ParseAllImages and ParseAllLinks functions ? :P



    I told you how to do that, yet you never gave credits ? :P
    I couldn't think of much else to include

    Quote Originally Posted by Jeremy S. Anderson
    There are only two things to come out of Berkley, Unix and LSD,
    and I don’t think this is a coincidence
    You can win the rat race,
    But you're still nothing but a fucking RAT.


    ++Latest Projects++
    [Open Source] Injection Library
    Simple PE Cipher
    FilthyHooker - Simple Hooking Class
    CLR Injector - Inject .NET dlls with ease
    Simple Injection - An in-depth look
    MPGH's .NET SDK
    eJect - Simple Injector
    Basic PE Explorer (BETA)

  12. #11
    Hassan's Avatar
    Join Date
    May 2010
    Gender
    male
    Location
    System.Threading.Tasks
    Posts
    4,764
    Reputation
    495
    Thanks
    2,137
    My Mood
    Dead
    Quote Originally Posted by Jason View Post
    I couldn't think of much else to include
    You dissapointed me son. There are tons and tons of thing you can include (not mentioning the one's of msn). I'll make parsing and scrapping library soon.
    Last edited by Hassan; 03-08-2011 at 11:36 PM.

  13. #12
    Jason's Avatar
    Join Date
    Apr 2010
    Gender
    male
    Location
    /dev/null
    Posts
    5,704
    Reputation
    918
    Thanks
    7,682
    My Mood
    Mellow
    Quote Originally Posted by Hassan View Post


    You dissapointed me son. There are tons and tons of thing you can include (not mentioning the one's of msn). I'll make parsing and scrapping library soon.
    The one's you said in MSN were deemed useless by me. Like I said in MSN, I've never EVER even formed a half thought of using one of those functions. So why spend time writing the function? This library allows me to scrape sites with steez and ease. If you can't do it...too bad for you. Although I do remember when I first wanted to scrape sites you told me to use a streamreader and go through the page line-by-line .

    Quote Originally Posted by "Hassan

    I'll make parsing and scrapping library soon.
    No, you won't. You'll say you will, fuck around for a while then never release anything. What happened to your 5 "epic releases that will pwn this section" and the Uber Advanced WebBrowser tutorial with GeckoFX?
    Last edited by Jason; 03-09-2011 at 12:10 AM.

    Quote Originally Posted by Jeremy S. Anderson
    There are only two things to come out of Berkley, Unix and LSD,
    and I don’t think this is a coincidence
    You can win the rat race,
    But you're still nothing but a fucking RAT.


    ++Latest Projects++
    [Open Source] Injection Library
    Simple PE Cipher
    FilthyHooker - Simple Hooking Class
    CLR Injector - Inject .NET dlls with ease
    Simple Injection - An in-depth look
    MPGH's .NET SDK
    eJect - Simple Injector
    Basic PE Explorer (BETA)

  14. #13
    GuGy's Avatar
    Join Date
    Sep 2010
    Gender
    male
    Location
    Croatia
    Posts
    288
    Reputation
    11
    Thanks
    91
    My Mood
    Yeehaw
    Quote Originally Posted by -Away View Post
    Very nice, But i never understand things like this, PRO things
    Yea me to...I know basic things...
    [IMG]https://i1141.photobucke*****m/albums/n585/gugy9/opaaa.gif[/IMG]









  15. #14
    aLcohoL_95's Avatar
    Join Date
    Feb 2010
    Gender
    male
    Location
    SatyRicon
    Posts
    685
    Reputation
    8
    Thanks
    291
    My Mood
    Cynical
    which obfuscator did u u se ?

    CANNIBAL CORPSE P0WNS


  16. #15
    Jason's Avatar
    Join Date
    Apr 2010
    Gender
    male
    Location
    /dev/null
    Posts
    5,704
    Reputation
    918
    Thanks
    7,682
    My Mood
    Mellow
    .NET Reactor 4.0.0 .

    Quote Originally Posted by Jeremy S. Anderson
    There are only two things to come out of Berkley, Unix and LSD,
    and I don’t think this is a coincidence
    You can win the rat race,
    But you're still nothing but a fucking RAT.


    ++Latest Projects++
    [Open Source] Injection Library
    Simple PE Cipher
    FilthyHooker - Simple Hooking Class
    CLR Injector - Inject .NET dlls with ease
    Simple Injection - An in-depth look
    MPGH's .NET SDK
    eJect - Simple Injector
    Basic PE Explorer (BETA)

Page 1 of 2 12 LastLast