Parsing links from a webpage.

[kibbles18]

so i made an email bot (a program that reads and emails emails stored in a txt file) and now i want to make a program that can open up a URL as a string and search for all domains/text in it as well ass visit all links on the page. how should i approach this?

[Lyoto Machida]

Use an webbrowser, make it redirect to that site and then:

Dim Links As HtmlSomethingCollection = WebBrowser1.Document.GetElementByTagName("a")

to see all the links:

For Each SHit As HtmlElement in Links
Listbox1.Items.Add(Shit.GetAtributte("href").ToStr ing) // to add the links to a listbox
Next

its something like that, i dont code vb for a long time ago..

[kibbles18]

so i need to use a webbrowser and navigate to a page and then what?

heres my code so fat [wont work]

Public Class Form1

    Dim email As HtmlElement

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

    End Sub
    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        Dim i As Integer
        WebBrowser1.Navigate(TextBox1.Text)
        Dim emails As System.Windows.Forms.HtmlElementCollection = WebBrowser1.Document.GetElementsByTagName(TextBox2.Text)
        Dim count As Integer = WebBrowser1.Document.GetElementsByTagName(TextBox2.Text).Count
        For i = 0 To count
            ListBox1.Items.Add(email.GetAttribute("href").ToString)
        Next
    End Sub
End Class

[Lyoto Machida]

Public Class Form1

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
 
        WebBrowser1.Navigate(TextBox1.Text)
       
    End Sub


' put this on the WebBRowser1_DocumentComplete ' Event

 Dim emails As System.Windows.Forms.HtmlElementCollection = WebBrowser1.Document.GetElementsByTagName(TextBox2.Text)
        
        For Each  email As HtmlElement in emails
            ListBox1.Items.Add(email.GetAttribute("href").ToString)
        Next

End Class

But that is to get the link , is that what you want'?

[kibbles18]

i want to find all emails with @gmail.com or @hotmail.com first and display them in the list box

[Lyoto Machida]

Button 1:

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
 
        WebBrowser1.Navigate(TextBox1.Text)
       
    End Sub

WebBrowser_DocumentComplete event:

 Dim emails As System.Windows.Forms.HtmlElementCollection = WebBrowser1.Document.GetElementsByTagName(TextBox2.Text)
        
        For Each  email As HtmlElement in emails
If email.GetAttribute("href").ToString.Contains("@") then
            ListBox1.Items.Add(email.GetAttribute("href").ToString)
end if
        Next

[kibbles18]

if i use "a" in txtbox2 it works. how can i make it so it will find emails even without links? like if the email did not contain a link? and can u explain "href" and "a" to me? why we use it?

[freedompeace]

use the following regex:

<([a-zA-Z0-9="]+)(href|onclick)="([^"']+)">*</\1>

Verify the data within group 3 as HTTP URI, or evaluate the javascript to see if its a link to something.

[Lyoto Machida]

use the following regex:

<([a-zA-Z0-9="]+)(href|onclick)="([^"']+)">*</\1>

Verify the data within group 3 as HTTP URI, or evaluate the javascript to see if its a link to something.

yeah, Or just make some checks? Dont you know to use the If function?

If Textbox1.Text.Length > 5 Then Msgbox("The texbox have more then 5characters")

Make your own checks o.O

[kibbles18]

but sometimes the emails in a website dont have a link so how to get those? they are valid emails as well

[sythe179]

this work?

For Each str As String In WebBrowser1.Document.All
            If str = ("*" & "@" & "*" & "." & "*") Then
                listbox1.items.add(str)
            End If
        Next

[kibbles18]

like if it was in plain text and it wasnt a link to anywhere, would it still work? and thanks ***** ill try that when i get back

[Blubb1337]

this work?

For Each str As String In WebBrowser1.Document.All
            If str = ("*" & "@" & "*" & "." & "*") Then
                listbox1.items.add(str)
            End If
        Next

That should not work at all.

[sythe179]

That should not work at all.

ahh....i see why...
whoah...realy need some sleep....47 hours....yup..time to sleep..
hey tablets...HERE I COME!....

[kibbles18]

so i can use

WebBrowser1.Document.All

and then search for a string?

its not working. im trying to get it to find an email thats not a link, just text. heres me code:

        For Each emailstr As String In WebBrowser1.Document.All.ToString
            If emailstr.Contains("@") & emailstr.Contains(".") Then
                ListBox1.Items.Add(emailstr)
            End If
        Next