Website Source Code

Ok, so I want to get the source code of a website so i can parse through it and get some info, but after extensive research on the internet I have not found anything constructive enough to help me get the job done. I know about WinSock so if it uses that then don't be hesitant to tell me how. So just to rephrase my inquiry for clarity: I want to get the source code of a website (html code) with a c++ program. Thanks in advance.

Hurr Durr you can't that's why there's a server processing the web language (such as PHP, ASP, etc.)

Originally Posted by HeXel

Hurr Durr you can't that's why there's a server processing the web language (such as PHP, ASP, etc.)

Thanks for your reply, but it's not true. You can do it in c++. So please, let others with experience answer. I wouldn't have asked unless i knew it was possible with c++.

You must send a HTTP GET request to the server you wish to get the website from. The server will send you it's index page, from here you can parse all links from the page. To find images for example you'd look for something like <img src="/source/to/image/goes/here.jpg" foo="bar" /> Wikipedia can help you sending a basic GET request (be sure to use the English page).

@HeXel

Those 'servers' as you call them are MADE with C and C++, php is based on C and is developed in C.

Originally Posted by .::SCHiM::.

You must send a HTTP GET request to the server you wish to get the website from. The server will send you it's index page, from here you can parse all links from the page. To find images for example you'd look for something like <img src="/source/to/image/goes/here.jpg" foo="bar" /> Wikipedia can help you sending a basic GET request (be sure to use the English page).

@HeXel

Those 'servers' as you call them are MADE with C and C++, php is based on C and is developed in C.

Thanks Schim, I am going to try it out and report back on how it went.

Originally Posted by .::SCHiM::.

@HeXel

Those 'servers' as you call them are MADE with C and C++, php is based on C and is developed in C.

Holy shit here is captain obvious, if you didn't tell me that I wouldn't know my master ...

Seriously say something useful. PHP != C++ in a lot of aspects, although it as the same syntax in some stuff.

Originally Posted by 258456

Thanks for your reply, but it's not true. You can do it in c++. So please, let others with experience answer. I wouldn't have asked unless i knew it was possible with c++.

Sorry I though you were one of those beggers asking for some stupid stuff. tl;dr OP.

You're right.

Originally Posted by 258456

EDIT:=========================

Ok so the most successful i came is to this point:

OP what you want to do with the HTML ?

Originally Posted by HeXel

Holy shit here is captain obvious, if you didn't tell me that I wouldn't know my master ...

Seriously say something useful. PHP != C++ in a lot of aspects, although it as the same syntax in some stuff.

Sorry I though you were one of those beggers asking for some stupid stuff. tl;dr OP.

You're right.

OP what you want to do with the HTML ?

I am going to parse the html code for info that i need to check for my app.

---------- Post added at 07:06 PM ---------- Previous post was at 07:05 PM ----------

Originally Posted by Fovea

Try "refreshing" the page (meta http-equiv="refresh") by sending the same GET request. Also, if the server was sending in chunks your code would fail to reproduce the chunks accurately because you overwrite the buffer each time. The only thing your code will produce is the last chunk.

True. Sorry for the failed logic, it was late last night, lol. I will fix it and try what you said and report back. Thanks.

---------- Post added at 07:15 PM ---------- Previous post was at 07:06 PM ----------

I have concluded that it isn't being sent in chunks because I initialized an int to 0 then made it increase by one everytime recv was being called and it returned 1. Meaning it wasn't being sent in chunks. Look, i don't what to do any more. All i want is to get the page source nothing more. Can you try to see what's the issue, or see if you can get the page source. The url is: freecfhacks.forum motion.com

Thx

Ok schim. I got it working with winsock. But i just get like header information sent back. How can i actually get the code of the page?

The page source is in the body of the response.

Originally Posted by Fovea

The page source is in the body of the response.

And how can I access the body of the page? I am sorry for so many questions, but I just don't want to learn the whole http protocol for something I am not going to use. Thanks for your responses.

The content is right after the header of the response. If you want concrete examples, use a packet sniffer or debug your web browser (hook recv and examine buffer after call).

well this is my code:

Code:

char buffer[50000];
	memset(buffer,0,999);
	send(Socket, "GET HTTP/1.1 \r\n Host: freecfhacks.forum(m)ot(i)o(n).com \r\n\r\n", 60, 0); // the site is without () i just wanted you to see so u can test
	
	int inDataLength=recv(Socket,buffer,50000,0);
	std::cout<<buffer;

And it works and i don't get the correct body. I checked my browser and hit view source and i don't get the same code. Maybe you can tell me what i am doing wrong. @Fovea

Code:

GET / HTTP/1.1\r\nHost: www.google.com\r\nConnection: close\r\n\r\n

Put your recv in a loop until the connection is closed because the server might send the response back in chunks. This occurs if the response you receive contains the header-value Transfer-Encoding: chunked.

It worked for google,but for my site it gives me a 301 Moved Permanently message.

Code:

char buffer[50000];
	memset(buffer,0,50000);
	send(Socket, "GET / HTTP/1.1\r\nHost: www.freecfhacks.***********.com\r\nConnection: close\r\n\r\n", strlen("GET / HTTP/1.1\r\nHost: www.freecfhacks.***********.com\r\nConnection: close\r\n\r\n"), 0);
	
	while(recv(Socket,buffer,6000,0) != 0 ){}
	
	std::cout<<buffer;

@Fovea , thanks for ur time btw.

EDIT:=========================

Ok so the most successful i came is to this point:

Try "refreshing" the page (meta http-equiv="refresh") by sending the same GET request. Also, if the server was sending in chunks your code would fail to reproduce the chunks accurately because you overwrite the buffer each time. The only thing your code will produce is the last chunk.

Website Source Code

Post a Reply

Similar Threads

Tags for this Thread